Methods, apparatuses and program recording media for image coding and image decoding

ABSTRACT

An object of this invention is to provide a coding method and a coding apparatus for efficiently coding both a pixel value signal and a shape signal which are included in an image signal to be coded, wherein a prediction changing unit outputs a reference pixel value changing signal and a reference shape signal to a switching circuit which selects a reference pixel value signal and to a switching circuit which selects a reference shape signal, respectively, so as to control the switching circuit in a way that appropriate reference signals are to be selected.

This application is a continuation of application Ser. No. 09/561,343filed Apr. 28, 2000, which is a continuation of application Ser. No.09/012,191 filed Jan. 23, 1998, now U.S. Pat. No. 6,088,485.

FIELD OF THE INVENTION

This invention relates to an image coding/decoding method, an imagecoding/decoding apparatus and an image coding/decoding program recordingmedium, and particularly, to image coding with fewer bit number andwithout deteriorating the picture quality, for recording andtransmitting image signals having information on the shape of an objectso as to process image signals for each object efficiently, and to imagedecoding for decoding the result of the image coding.

BACKGROUND OF THE INVENTION

The technology for digitizing images into digital image data isdramatically spreading and developing because digital data is easy torecord, transmit, edit, copy and transfer. One of the advantages ofdigitization is the possibility of facilitating data compression.Compression coding is an important technology for data recording anddata transmission. The compression coding technology has the establishedinternational standards, especially one of which is the MPEG standardthat has spread as a general digital standard which can process bothvideo and audio.

The compression coding of digital images processes image data comprisinga series of digitized still pictures. In general, the compression codinghas the two ways, one of which is an intra-frame coding which compressesa frame (corresponding to a picture) of still picture according to thespatial correlation (the correlation in a frame) while removingredundancy, and the other of which is a inter-frame coding whichcompresses frames of still pictures which are temporally close to eachother, for example, temporally serial frames of pictures, according tothe temporal correlation (the correlation among frames) while removingredundancy.

A prior art image coding based on MPEG and the like usually uses theintra-frame coding. If the inter-frame coding is carried out as well,the coded data has a high compression rate. To carry out the inter-framecoding, a decoding process which is a converse process of coding andmotion detection and motion compensation processes are carried out togenerate a predicted image and then a difference between an image to becoded and the predicted image is calculated using the predicted image asa reference image. Thus the decoding process, and the motion detectionand motion compensation processes adversely increase the process loadfor an apparatus. However, the difference is small when the predictedimage has preferable precision, it is possible to increase the codingefficiency by coding the difference more than by coding the image to becoded itself.

As the prediction method employed when the inter-frame coding arecarried out, there are some methods, namely a forward prediction basedon data which is located at a forward position on time series from thedata of an image to be coded in a series of still pictures, a backwardprediction based on data which is located at a backward position, and abidirectional prediction based on data which are located at a forwardand backward positions. In general, the intra-frame coding isrepresented as ‘I’, the forward predictive coding is represented as ‘P’,and the bidirectional predictive coding (including the backwardpredictive coding) is represented as ‘B’.

When only the intra-frame coding is carried out, or when the forwardpredictive coding as well as the intra-frame coding are carried out, aseries of still pictures to be coded can be processed simply accordingto time series. As opposed to this, when the backward or bidirectionalprediction is carried out, the data which is located at a backwardposition on time series must be first coded. Therefore, in general, whenthe inter-frame coding is carried out as well, it is determined inadvance which each frame constituting the image data to be coded, an Iframe to be subjected to the intra-frame coding or a P frame which canbe subjected to the forward predictive coding or a B frame which can besubjected to the bidirectional predictive coding. If the data to beprocessed is an I frame, the data is subjected to the intra-framecoding. If the data to be processed is a P frame or B frame, the data issubjected to the intra-frame coding or the inter-frame coding. When thiscoding process is carried out, it is possible to predetermine the ratioof the I frame and the P frame and the B frame according to the purposeof the result of the coding and to the like in the coding apparatus.

FIG. 14 is a diagram for explaining the intra- and inter-frame codingprocesses of the prior art. In the figure, numerals 1400 to 1406 eachdesignate a frame of image data constituting an image data to be coded.Numerals t0 to t6 designate the respective times. The order of the timest0 to t6 indicates the course of time series. In the frames 1400 to1406, the frame 1400 is an T frame, the frames 1403 and 1406 are Pframes, and the frames 1401, 1402, 1404 and 1405 are B frames.

Arrows shown in the figure designate the reference relationships of eachframe in the coding process. The frame 1400 which is an I frame issubjected to the intra-frame coding without referring to any otherframe. The frame 1403 which is a P frame can be coded referring to theframe 1400 which is located at a forward position on time series. Theframe 1401 which is a B frame can be coded referring to the frame 1400which is located at a forward position on time series and/or the frame1403 which is located at a backward position on time series.

For that reason, as described above, the frame 1403 must be codedearlier than the frames 1401 and 1402 which are located at a forwardposition in the frame 1403, and the I frame and the P frame are givenpriority to be coded earlier than the B frame. Further, no frames arecoded referring to the B frame.

When the bidirectional predictive coding is additionally carried out forthe coding process, the apparatus can decide whether the B frame issubjected to the inter-frame coding referring to a forward and backwardframes, or the B frame is subjected to either a forward frame, abackward frame, or both frames which are selected as reference frames,or the intra-frame coding is an option as well.

As described above, the inter-frame coding, particularly when thebidirectional predictive coding is carried out as well, contributes toan increase in the process load and requires a storage means which has alarge memory capacity for retaining temporally adjacent data. However,prediction with high-level precision makes a difference between apredicted image which is obtained by the prediction and an image to becoded small, whereby coding efficiency can be improved. Thus the codingmethod is determined according to the performance of an apparatus, thepicture quality, the properties of coded data to be required and so on.

On the other hand, a method for coding image signals for each object hasoften been used in recent years. ISO standardizes these method as MPEG4.In November 1996, what is called the video verification model VM5.0 wasworked out. The image signal for each object consists of pixel valuesignals which indicate brightness and color and are called texture andshape signals which represent the shape of the object. The image signalhaving this form is being utilized most in the computer graphicstechnology, and in the field where image sources are created such as thedepartment of producing programs.

FIG. 15(a) to FIG. 15(c) are diagrams for explaining the coding for eachobject in the prior art. FIG. 16(a) and FIG. 16(b) are diagrams forexplaining a signal processing for the coding for each object. FIG.15(a) shows an example of objects to be coded, which is an imageconsisting of a background image and a foreground image (a goldfishswimming in a fish tank). FIG. 15(b) shows the foreground (thegoldfish). FIG. 15(c) shows the background (water plants and water inthe fish tank).

To composite the foreground image and the background image, informationwhich is used for deciding which pixel constituting the composite imagerepresents the foreground or the background, is required. For thisreason, the foreground image shown in FIG. 15(b) consists of the pixelvalue signal shown in FIG. 16(a) and the shape signal (a binary alphasignal) shown in FIG. 16(b), the shape signal specifying the imagerepresentation. In this case, the pixel value signal indicates thetexture of the goldfish and includes the brightness signal and colorsignal of each pixel. The shape signal indicates the profile of thegoldfish, i.e. the contour of the goldfish, and is a two-valued signalhaving a value ‘1’ inside the contour or a value ‘0’ outside thecontour. This shape signal indicates the foreground in the compositionof the image. The shape signal shows that, in the figure, the regionindicated by the black part has the value ‘1’ and represents theforeground. In general, when the coding is carried out for each object,the pixel values signal and the shape signal are applied to specifiedobjects while only the pixel value signal is applied to the parts otherthan the specified objects, whereby the coding efficiency is improved.As described above, in this case, the goldfish, i.e. the foregroundimage, is processed as a specified object.

The efficiency of coding the pixel value signal shown in FIG. 16(a) isimproved because the pixel value signal shown in FIG. 16(a) is codedbased on the above-mentioned temporal correlation referring to thesignal which is obtained by decoding a pixel value signal which has beencoded. There is another coding method which makes the coding efficiencymore higher by adaptively changing two images for reference than byreferring the pixel value signal of one image. The standards such as ISOMPEG1/2 and ITU-T H.261 provide the coding method which refers twoimages.

FIG. 17(a) to FIG. 17(c) and FIG. 18(a) to FIG. 18(c) are diagrams forexplaining the coding of pixel value signals which refers a plurality ofpictures. FIG. 17(a) to FIG. 17(c) show the pixel value signals of theinput image which constitute the foreground image. FIG. 17(a) is takenat time t0. FIG. 17(b) is taken at time t1. FIG. 17(c) is taken at timet2. As shown in the figures, the three input pixel value signals arearranged in the same time series similarly to FIG. 14. A signal at timet0 is located at a forward position on time series from a signal at timet1. A signal at time t2 is located at a backward position on time seriesfrom a signal at time t1. The pixel value signal of the input image attime t1 shown in FIG. 17(b) has correlation with the pixel value signalat time t0 shown in FIG. 17(a) and the pixel value signal at time t2shown in FIG. 17(c).

FIG. 18(a) and FIG. 18(c) shows decoded pixel value signals which areobtained by decoding the pixel value signals shown in FIG. 17(a) andFIG. 17(c) which have been coded. The predicted image at time t1 shownin FIG. 18(b) is generated with good precision from the pixel valuesignals of the decoded images at time 0 and time 2 based on thecorrelation shown in FIG. 17(a) to FIG. 17(c).

The typical method of predicting images can generate a predicted imageat time t1 by motion-compensating already decoded images at time t1 andtime t2 and averaging them. As there is a strong correlation between thepredicted image at time t1 and the input image at time t1, the inputimage at time t1 is coded referring to the predicted image at time t1.That is, a difference image between the predicted image generated basedon the forward and backward images on time series and the input image iscalculated and then the pixel value signal of the difference image iscoded.

Thus, when the image to be coded has strong correlation with the imageslocated at a forward and backward positions on time series, it can beexpected that the prediction has better precision by utilizing both theforward and backward images than by utilizing either of them. If theprediction has good precision, the pixel value signal of the differenceimage has a small amount of data, whereby the coding with highefficiency can be realized.

As described above, in the case of coding images for each object, theefficiency of coding the pixel value signal is realized based on thetemporal correlation. On the other hand, the shape signal accompanyingthe pixel value signal is processed similarly to the pixel value signal,when only the intra-frame coding is carried out, or when the inter-framecoding accompanied with only the forward prediction is carried out.However, when the inter-frame coding accompanied with the bidirectionalprediction is carried out, a problem arises whereby the efficiency ofthe coding of the shape signal decreases if the shape signal isprocessed in a similar way to that for the pixel value signal.

As the pixel value signal is a multivalued signal and includes abrightness signal and a color signal, the possibility of obtaining thepreferable predicted image is strong because of the calculation ofobtaining the average as described above. Therefore, the codingefficiency is improved if the temporally adjacent data are retained andare subjected to the calculation such as obtaining a difference orobtaining an average. As opposed to this, in the case of the two-valuedshape signal as described above, for example, there is little merit evenif an average is calculated referring to plural piece of referenceinformation in order to obtain the preferable predicted image, becauseeither of the two values must be used when the obtained average isneither of the two values. In general, for the two-valued shape signal,because the temporally adjacent data are retained and are subjected tothe process such as obtaining an average, the precision of theprediction is not necessarily improved, but the utilization of theresource of the apparatus is prevented, or the coding efficiency isdecreased.

In a prior art image coding, when the shape signal as well as the pixelvalue signal are similarly processed, a problem arises whereby thecoding process with the bidirectional prediction decreases the processefficiency as described above. Thus, techniques for improving theefficiency of coding the pixel value signal are not simply applied tothe coding of the shape signal. For that reason, in some cases the shapesignal is processed by means of a method such as a reversiblecompression coding for two-valued signal which is used in a facsimileand the like, that is, the shape signal is recorded and transmittedapart from the pixel value signal in the prior art. However, areversible method has generally less efficiency than a irreversiblemethod, so the coding efficiency or the process efficiency would not bemuch improved.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an image codingmethod for coding the pixel value signal and the shape signal referringto the reference image, thereby improving the coding efficiency for boththe pixel value signal and the shape signal.

It is another object of the present invention to provide an image codingapparatus for coding the pixel value signal and the shape signalreferring to the reference image, thereby improving the codingefficiency for both the pixel value signal and the shape signal.

It is still another object of the present invention to provide an imagedecoding method for decoding the result of the coding which have beenefficiently coded by the image coding method.

It is yet another object of the present invention to provide an imagedecoding apparatus for appropriately decoding the result of the codingwhich have been efficiently coded by the image coding apparatus.

It is a further object of the present invention to provide a recordingmedium for recording an image coding program of coding the pixel valuesignal and the shape signal referring to the reference image, therebyimproving the coding efficiency for both the pixel value signal and theshape signal.

It is a still further object of the present invention to provide arecording medium for recording an image decoding program ofappropriately decoding signals which have been efficiently coded by theimage coding program.

Other objects and advantages of the present invention will becomeapparent from the detailed description desired hereinafter; it should beunderstood, however, that the detailed description and specificembodiment are desired by way of illustration only, since variouschanges and modifications within the scope of the invention will becomeapparent to those skilled in the art from this detailed description.

A conception that the shape signal is subjected to a prediction processadapting to the property of the shape signal by controlling theselection of a reference image used in the prediction process based onthe temporal correlation independently of the pixel value signal, isutilized to obtain the objects.

According to a first aspect of the present invention, an image codingmethod for coding an input image signal including a shape signalindicating the shape of an object and a pixel value signal havinginformation on the color and brightness of the object, comprising apixel value signal coding step for coding the pixel value signalincluded in the input image signal referring to a decoded pixel valuesignal obtained by decoding a pixel value signal which has been alreadycoded; a shape signal coding step for coding the shape signal includedin the input image signal referring to a decoded shape signal obtainedby decoding a shape signal which has been already coded; and a codingreference specification signal generating step for generating areference pixel value specification signal for specifying the decodedpixel value signal to be referred in the pixel value signal coding stepand a reference shape specification signal for specifying the decodedshape signal to be referred in the shape signal coding step, and then,based on the generated signals, generating a prediction selection signalhaving information indicating a reference method in a coding process,whereby the coding in the pixel value signal coding step and the shapesignal coding step is carried out referring to the respectively selectedreference signal.

According to a second aspect of the present invention, the image codingmethod of the first aspect wherein in the shape signal coding step, aforward time decoded shape signal which is obtained from a shape signalwhich is located at a forward position on time series from a shapesignal to be coded, and a backward time decoded shape signal which isobtained from a shape signal which is located at a backward position ontime series from the shape signal to be coded, are used to be referredas the decoded shape signal, whereby a shape signal is coded using acorrelation between the shape signal and its temporally adjacentsignals.

According to a third aspect of the present invention, the image codingmethod of the second aspect wherein in the pixel value signal codingstep, a forward time decoded pixel value signal which is obtained from apixel value signal which is located at a forward position on time seriesfrom a pixel value signal to be coded, and a backward time decoded pixelvalue signal which is obtained from a pixel value signal which islocated at a backward position on time series from the pixel valuesignal to be coded, are used to be referred as the decoded pixel valuesignal, whereby the a shape signal and a pixel value signal are codedusing a correlation between the signals and their temporally adjacentsignals.

According to a fourth aspect of the present invention, the image codingmethod of the first aspect wherein in the coding reference specificationsignal generating step, when the prediction selection signal isgenerated, the reference pixel value specification signal and thereference shape specification signal are integrated and coded, wherebythe prediction selection signal is generated by assigning a short codelength to a frequent phenomenon.

According to a fifth aspect of the present invention, the image codingmethod of the first aspect wherein in the coding reference specificationsignal generating step, when the prediction selection signal isgenerated, a method for indicating the specification of a decoded pixelvalue signal to be referred is determined corresponding to a method forindicating the specification of a decoded shape signal, whereby theprediction selection signal is generated by assigning a short codelength to a frequent phenomenon.

According to a sixth aspect of the present invention, the image codingmethod of the first aspect wherein in the coding reference specificationsignal generating step, the reference decoded pixel value signal isspecified for each pixel value changing unit which is a unit of theinput pixel value signal, and the reference decoded shape signal isspecified for each shape changing unit which is a unit of the inputshape signal, whereby the coding process is carried out according to theproperty of the shape signal while reducing the changing frequencywithout any serious reduction in prediction precision.

According to a seventh aspect of the present invention, the image codingmethod of the sixth aspect wherein the shape changing unit is a frameconstituting the input shape signal, whereby the coding process iscarried out by means of the changing for each frame unit according tothe property of the shape signal while reducing the changing frequencywithout any serious reduction in prediction precision.

According to an eighth aspect of the present invention, the image codingmethod of the sixth aspect wherein the changing unit is a hierarchicalunit containing a large block unit constituting the input shape signaland a small block unit constituting the large block, whereby the codingprocess is carried out by means of the changing for each hierarchicalunit according to the property of the shape signal while reducing thechanging frequency without any serious reduction in predictionprecision.

According to a ninth aspect of the present invention, the image codingmethod of the first aspect wherein the shape signal coding step includesa comparison judgment step wherein among a forward time decoded shapesignal which is obtained from a shape signal which is located at aforward position on time series from a shape signal to be coded and abackward time decoded shape signal which is obtained from a shape signalwhich is located at a backward position on time series from the shapesignal to be coded, the decoded shape signal more adjacent to the shapesignal to be coded is selected, and the decoded shape signal selected inthe comparison judgment step is referred, whereby the reference processin the shape signal coding process is much simplified according to theproperty of the shape signal without any serious reduction in predictionprecision.

According to a tenth aspect of the present invention, the image codingmethod of the first aspect wherein in the shape signal coding step, aforward time decoded shape signal which is obtained from a shape signalwhich is located at a forward position on time series from a shapesignal to be coded is referred, whereby the reference process in theshape signal coding process is much simplified according to the propertyof the shape signal without any serious reduction in predictionprecision.

According to an eleventh aspect of the present invention, the imagecoding method of the first aspect wherein the shape signal coding stepincludes a comparison judgment step wherein among a forward time decodedshape signal which is obtained from a shape signal which is located at aforward position on time series from a shape signal to be coded and abackward time decoded shape signal which is obtained from a shape signalwhich is located at a backward position on time series from the shapesignal to be coded, the decoded shape signal more adjacent to the shapesignal to be coded is selected, and includes a forward fixed stepwherein the forward time decoded shape signal is selected, and thedecoded shape signal selected in the comparison judgment step or theforward fixed step is referred, whereby the reference process in theshape signal coding process is much simplified by means of the changingaccording to the property of the shape signal without any seriousreduction in prediction precision.

According to a twelfth aspect of the present invention, an imagedecoding method which decodes a coded shape signal and a coded pixelvalue signal which are obtained by coding a shape signal which isincluded in an image signal and indicates the shape of an object and apixel value signal which is included in the image signal and hasinformation on the color and brightness of the object, comprising adecoding reference specification signal generating step for generating areference pixel value specification signal which specifies a decodedpixel value signal to be referred in decoding the coded pixel valuesignal, and a reference shape specification signal which specifies adecoded shape signal to be referred in decoding the coded shape signal,based on information obtained from a prediction selection signalincluding information indicating a reference method in a coding process,using the prediction selection signal; a pixel value signal decodingstep for decoding the coded pixel value signal referring to a decodedpixel value signal specified based on the reference pixel valuespecification signal generated by the decoding reference specificationsignal generating step; and a shape signal decoding step for decodingthe coded shape signal referring to a decoded shape signal specifiedbased on the reference shape specification signal generated by thedecoding reference specification signal generating step, whereby in thepixel value signal decoding step and in the shape signal decoding stepthe decoding is carried out using a reference appropriate for each step.

According to a thirteenth aspect of the present invention, the imagedecoding method of the twelfth aspect wherein in the shape signaldecoding step, a forward time decoded shape signal obtained from a shapesignal which is located at a forward position on time series from thecoded shape signal to be decoded and a backward time decoded shapesignal obtained from a shape signal which is located at a backwardposition on time series from the coded shape signal to be decoded areused as the decoded shape signal to be referred, whereby the coded shapesignal is decoded referring to its temporally adjacent signal.

According to a fourteenth aspect of the present invention, the imagedecoding method of the thirteenth aspect wherein in the pixel valuesignal decoding step, a forward time decoded pixel value signal obtainedfrom a pixel value signal which is located at a forward position on timeseries from the coded pixel value signal to be decoded and a backwardtime decoded pixel value signal obtained from a pixel value signal whichis located at a backward position on time series from the coded pixelvalue signal to be decoded are used as the decoded pixel value signal tobe referred, whereby the coded pixel value signal and the coded pixelvalue signal are decoded referring to their temporally adjacent signals.

According to a fifteenth aspect of the present invention, the imagedecoding method of the twelfth aspect wherein in the decoding referencespecification signal generating step, the reference pixel valuespecification signal and the reference shape specification signal whichhave been integrated and coded are obtained by decoding the predictionselection signal, whereby the prediction selection signal which has beencoded by assigning a short code length to a frequent phenomenon isdecoded.

According to a sixteenth aspect of the present invention, the imagedecoding method of the twelfth aspect wherein in the decoding referencespecification signal generating step, a method for indicating thespecification of the decoded pixel value signal to be referred isdetermined in a decoding process of the prediction selection signalaccording to the method for indicating the specification of the decodedshape signal to be referred, whereby the prediction selection which hasbeen coded by assigning a short code length to a frequent phenomenon isdecoded.

According to a seventeenth aspect of the present invention, the imagedecoding method of the twelfth aspect wherein in the decoding referencespecification signal generating step, the reference decoded pixel valuesignal is specified for each pixel value changing unit which is a unitof the coded pixel value signal and the reference decoded shape signalis specified for each shape changing unit which is a unit of the codedshape signal, whereby the coded shape signal which has been coded withthe less frequent changing is decoded.

According to an eighteenth aspect of the present invention, the imagedecoding method of the sixteenth aspect wherein the shape changing unitis a frame constituting the coded shape signal, whereby the coded shapesignal which has been coded with the less frequent changing is decodedby carrying out the changing for each frame.

According to a nineteenth aspect of the present invention, the imagedecoding method of the sixteenth aspect wherein the changing unit is ahierarchical unit containing a large block unit constituting the inputshape signal and a small block unit constituting the large block,whereby the coded shape signal which has been coded with the lessfrequent changing is carried out by means of the changing for eachhierarchical unit.

According to a twentieth aspect of the present invention, the imagedecoding method of the twelfth aspect wherein the shape signal decodingstep includes a comparison judgment step wherein among a forward timedecoded shape signal which is obtained from a shape signal which islocated at a forward position on time series from the coded shape signalto be decoded and a backward time decoded shape signal which is obtainedfrom a shape signal which is located at a backward position on timeseries from the coded shape signal to be decoded, the decoded shapesignal more adjacent to the coded shape signal to be decoded isselected, and the decoded shape signal selected in the comparisonjudgment step is referred, whereby the coded shape signal which has beensubjected to the coding process in which the reference process issimplified is decoded.

According to a twenty-first aspect of the present invention, the imagedecoding method of the twelfth aspect wherein in the shape signaldecoding step, a forward time decoded shape signal which is obtainedfrom a shape signal which is located at a forward position on timeseries from the coded shape signal to be decoded is referred, wherebythe coded shape signal which has been subjected to the coding process inwhich the reference process is simplified is decoded.

According to a twenty-second aspect of the present invention, the imagedecoding method of the twelfth aspect wherein the shape signal decodingstep includes a comparison judgment step wherein among a forward timedecoded shape signal which is obtained from a shape signal which islocated at a forward position on time series from the coded shape signalto be decoded and a backward time decoded shape signal which is obtainedfrom a shape signal which is located at a backward position on timeseries from the coded shape signal to be decoded, the decoded shapesignal more adjacent to the coded shape signal to be decoded isselected, and includes a forward fixed step wherein the forward timedecoded shape signal is selected, and the decoded shape signal selectedin the comparison judgment step or the forward fixed step is referred,whereby the coded shape signal which has been subjected to the codingprocess in which the reference process is simplified is decoded.

According to a twenty-third aspect of the present invention, an imagecoding apparatus for coding an input image signal including a shapesignal indicating the shape of an object and a pixel value signal havinginformation on the color and brightness of the object, comprising apixel value signal coding means for coding the pixel value signalincluded in the input image signal referring to a decoded pixel valuesignal obtained by decoding a pixel value signal which has been alreadycoded; a shape signal coding means for coding the shape signal includedin the input image signal referring to a decoded shape signal obtainedby decoding a shape signal which has been already coded; and a codingreference specification signal generating means for generating areference pixel value specification signal for specifying the decodedpixel value signal to be referred in the pixel value signal coding stepand a reference shape specification signal for specifying the decodedshape signal to be referred in the shape signal coding step and then,based on the generated signals, generating a prediction selection signalhaving information indicating a reference method in a coding process,whereby the coding in the pixel value signal coding step and the shapesignal coding step is carried out referring to the respectively selectedreference signal.

According to a twenty-fourth aspect of the present invention, an imagedecoding apparatus which decodes a coded shape signal and a coded pixelvalue signal which are obtained by coding a shape signal which isincluded in an image signal and indicates the shape of an object and apixel value signal which is included in the image signal and hasinformation on the color and brightness of the object, comprising adecoding reference specification signal generating means for generatinga reference pixel value specification signal which specifies a decodedpixel value signal to be referred in decoding the coded pixel valuesignal and a reference shape specification signal which specifies adecoded shape signal to be referred in decoding the coded shape signal,based on information obtained from a prediction selection signalincluding information indicating a reference method in a coding process,using the prediction selection signal; a pixel value signal decodingmeans for decoding the coded pixel value signal referring to a decodedpixel value signal specified based on the reference pixel valuespecification signal generated by the decoding reference specificationsignal generating step; and a shape signal decoding means for decodingthe coded shape signal referring to a decoded shape signal specifiedbased on the reference shape specification signal generated by thedecoding reference specification signal generating step, whereby in thepixel value signal decoding step and in the shape signal decoding stepthe decoding is carried out using a reference appropriate for each step.

According to a twenty-fifth aspect of the present invention, an imagecoding program recording medium for recording an image coding programfor coding an input image signal including a shape signal indicating theshape of an object and a pixel value signal having information on thecolor and brightness of the object, the program comprising a pixel valuesignal coding step for coding the pixel value signal included in theinput image signal referring to a decoded pixel value signal obtained bydecoding a pixel value signal which has been already coded; a shapesignal coding step for coding the shape signal included in the inputimage signal referring to a decoded shape signal obtained by decoding ashape signal which has been already coded; and a coding referencespecification signal generating step for generating a reference pixelvalue specification signal for specifying the decoded pixel value signalto be referred in the pixel value signal coding step and a referenceshape specification signal for specifying the decoded shape signal to bereferred in the shape signal coding step, and then, based on thegenerated signals, generating a prediction selection signal havinginformation indicating a reference method in a coding process, wherebythe coding in the pixel value signal coding step and the shape signalcoding step is carried out referring to the respectively selectedreference signal by executing the image coding program on a computersystem and the like.

According to a twenty-sixth aspect of the present invention, an imagedecoding program recording medium for recording an image decodingprogram which decodes a coded shape signal and a coded pixel valuesignal which are obtained by coding a shape signal which is included inan image signal and indicates the shape of an object and a pixel valuesignal which is included in the image signal and has information on thecolor and brightness or the object, the program comprising a decodingreference specification signal generating step for generating areference pixel value specification signal which specifies a decodedpixel value signal to be referred in decoding the coded pixel valuesignal and a reference shape specification signal which specifies adecoded shape signal to be referred in decoding the coded shape signal,based on information obtained from a prediction selection signalincluding information indicating a reference method in a coding process,using the prediction selection signal; a pixel value signal decodingstep for decoding the coded pixel value signal referring to a decodedpixel value signal specified based on the reference pixel valuespecification signal generated by the decoding reference specificationsignal generating step; and a shape signal decoding step for decodingthe coded shape signal referring to a decoded shape signal specifiedbased on the reference shape specification signal generated by thedecoding reference specification signal generating step, whereby in thepixel value signal decoding step and in the shape signal decoding stepthe decoding is carried out using a reference appropriate for each stepby executing the image decoding program on a computer system and thelike.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of an image codingapparatus according to a first embodiment of the present invention.

FIGS. 2(a) to (c) are diagrams for explaining a shape signal codingprocess according to the first embodiment.

FIG. 3 is a block diagram showing the configuration of an image decodingapparatus according to a second embodiment of the present invention.

FIG. 4(a) and FIG. 4(b) are tables for explaining the code assignment ofa prediction selection signal generating process according to a thirdembodiment of the present invention.

FIG. 5 is a table for explaining the code assignment of a predictionselection signal generating process according to a fourth embodiment ofthe present invention.

FIG. 6(a) and FIG. 6(b) are tables for explaining the code assignment ofa prediction selection signal generating process according to a fifthembodiment of the present invention.

FIG. 7(a) and FIG. 7(b) are diagrams for explaining the changing unit ora coding process according to a ninth embodiment.

FIG. 8 is a diagram for explaining the changing unit of a shape signalcoding process according to a tenth embodiment of the present invention.

FIG. 9(a) and FIG. 9(b) are diagrams for explaining the referencerelationship of a coding process according to a thirteenth embodiment ofthe present invention.

FIG. 10 is a flow chart showing the procedure of a coding process for aB frame according to the thirteenth embodiment.

FIG. 11 is a flow chart showing the procedure of a coding control of aprediction change unit according to the thirteenth embodiment.

FIG. 12 is a flow chart showing the procedure of a decoding process fora B frame of an image decoding apparatus according to a sixteenthembodiment of the present invention.

FIG. 13 is a diagram showing a floppy disk employed as an image codingprogram recording medium according to a nineteenth embodiment of thepresent invention and as an image decoding program recording mediumaccording to a twentieth embodiment of the present invention.

FIG. 14 is a diagram for explaining an image coding process based ontemporal correlation in a prior art.

FIG. 15(a) to FIG. 15(c) are diagrams for explaining an image codingprocess carried out for each object in a prior art.

FIG. 16(a) and FIG. 16(b) are diagrams for explaining a pixel value andshape signal in an image coding process carried out for each object in aprior art.

FIG. 17(a) to FIG. 17(c) are diagrams for explaining the temporalrelationship of pixel value signals in an image coding process carriedout for each object in a prior art.

FIG. 18(a) to FIG. 18(c) are diagrams for explaining an image codingprocess carried out for each object based on the temporal relationshipof pixel value in a prior art.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1

The object of an image coding method and image coding apparatusaccording to a first embodiment of the present invention is to improvecoding efficiency by changing the respective reference signals for apixel value signal and a shape signal.

FIG. 1 is a block diagram showing the configuration of the image codingapparatus according to the first embodiment. As shown in the figure, theimage coding apparatus according to the first embodiment comprises asubtracter 101, an encoder (for pixel value signals) 102, a decoder (forpixel value signals) 103, an adder 104, a first switching circuit 105, asecond switching circuit 106, a memory (a first memory for retainingdecoded pixel value signals) 107, a memory (a second memory forretaining decoded pixel value signals) 108, an average calculator 109, aprediction changing unit 110, a encoder (for shape signals) 111, adecoder (for shape signals) 112, a third switching circuit 113, a fourthswitching circuit 114, a memory (a first memory for retaining decodedshape signals) 115, and a memory (a second memory for retaining decodedshape signals) 116.

In the figure, the subtracter 101 calculates a difference between aninput pixel value signal S151 which is the input into the image codingapparatus and a reference pixel value signal S155 which is output fromthe second switching circuit 106 described hereinafter, to generate adifference pixel value signal S152. The encoder (for pixel valuesignals) 102 subjects the difference pixel value signal S152 to acompression coding to generate a coded pixel value signal S153. Thedecoder (for pixel value signals) 103 subjects the coded pixel valuesignal S153 to a decoding process which is the converse process of thecoding process to generate a coded and then decoded pixel value signalS154. The adder 104 adds the coded and then decoded pixel value signalS154 to a reference pixel value signal S155, which is output by thesecond switching circuit 106, to generate a decoded pixel value signalS156.

The first switching circuit 105 switches the output destinations for thedecoded pixel value signal S156 according to a decoding pixel valuechanging signal S171 input from the prediction changing unit 110. Theswitching of the first switching circuit 105 decides either that thedecoded pixel value signal S156 is input and stored in the memory 107 orthe memory 108, or that the decoded pixel value signal S156 is inputneither the memories.

The second switching circuit 106 selects a signal, which is to be usedas the reference pixel value signal S155, according to a reference pixelvalue changing signal S172 input from the prediction changing unit 110.The selection of the second switching circuit 106 decides either a firstdecoded and then stored pixel value signal S157 stored in the memory107, a second decoded and then stored pixel value signal S158 stored inthe memory 108, an average decoded pixel value signal S159 obtained bythe average calculator 109, or a predetermined value, to be used as thereference pixel value signal S155. In this case, the predetermined valueis fixed pixel data which is used when the encoder 102 carries out theintra-frame coding. The selected reference pixel value signal S155 isoutput to the subtracter 101 and to the adder 104.

The memories 107 and 108 retain the decoded pixel value signal S156input from the first switching circuit 105 in frames. The averagecalculator 109 receives temporally different frames of the decoded pixelvalue signals stored in the memories 107 and 108 by the switching of thefirst switching circuit 105, and obtains the average of the decodedpixel value signals to generate the average decoded pixel value signalS159. In the first embodiment, the decoded pixel value signal stored inthe memories 107 and 108 is either a forward time decoded pixel valuesignal obtained from a pixel value signal which is located at a forwardposition on time series from a pixel value signal to be coded, or abackward time decoded pixel value signal obtained from a pixel valuesignal which is located at a backward position from a pixel value signalto be coded.

The subtracter 101, the encoder 102, the decoder 103, the adder 104, andthe average calculator 109, the memory 107, the memory 108, the firstswitching circuit 105 and the second switching circuit 106 serve as apixel value signal coding means which codes the input pixel value signalwith reference to a pixel value signal which has been decoded.

The prediction changing unit 110 outputs the decoding pixel valuechanging signal S171, the reference pixel value changing signal S172,the decoding shape changing signal S173 and the reference shape changingsignal S174, which are control signals for the first to fourth switchingcircuit, to control the changing of the output destination of thedecoded pixel value signal, the selection of the reference pixel valuesignal, the changing of the output destination of the decoded shapesignal and the selection of reference shape signal, in the respectiveswitching circuits. Also, the prediction changing unit 110 serves as acoding reference specification signal generating means for generatingthe prediction selection signal S175 which is obtained by coding theresult of the output of the reference pixel value changing signal usedas the reference pixel value specification signal specifying thereference pixel value signal and the reference shape changing signalused as the reference shape specification signal specifying thereference shape signal.

The encoder (for shape signals) 111 subjects an input shape signal S161,which is the input into the image coding apparatus, to a compressioncoding which is carried out referring to a reference shape signal S166output from the fourth switching circuit 114 described hereinafter togenerate a coded shape signal S162. The decoder (for shape signals) 112subjects the coded shape signal S162 to a decoding process which is theconverse process of the coding process with reference to the referenceshape signal S166 output from the fourth switching circuit 114 togenerate a decoded shape signal S163.

The third switching circuit 113 switches the output destinations for thedecoded shape signal S163 according to a decoding shape changing signalS173 input from the prediction changing unit 110. The switching of thethird switching circuit 113 decides either that the decoded shape signalS163 is input and stored in the memory 115 or the memory 116, or thatthe decoded shape signal S163 is input neither the memories.

The fourth switching circuit 114 selects a signal, which is to be usedas the reference shape signal S166, according to a reference shapechanging signal S174 input from the prediction changing unit 110. Theselection of the fourth switching circuit 114 decides whether to useeither a first decoded and then stored shape signal S164 retained in thememory 115, a second decoded and then stored shape signal S165 retainedin the memory 116, or a predetermined value, as the reference shapesignal S166. The selected reference shape signal S166 is output to theencoder 111 and to the decoder 112.

The memories 115 and 116 retain the decoded shape signal S163 input fromthe third switching circuit 113 in frames. In the first embodiment, thedecoded shape signal stored in the memories 115 and 116 is either aforward time decoded shape signal obtained from a shape signal which islocated at a forward position from a shape signal to be coded, or abackward time decoded shape signal obtained from a shape signal which islocated at a backward position from the shape signal to be coded.

The encoder 111, the decoder 112, the memories 115 and 116, the thirdswitching circuit 113, and the fourth switching circuit 114 serve as ashape signal coding means which codes a shape signal referring to adecoded shape signal.

FIG. 2(a) to FIG. 2(c) are diagrams for explaining the coding of a shapesignal by the image coding apparatus according to the first embodiment.The operation of the image coding apparatus according to the firstembodiment is described as follows, referring to FIG. 1 and FIG. 2(a) toFIG. 2(c).

When the input pixel value signal S151 and the input shape signal S161are input into the image coding apparatus according to the firstembodiment, the input pixel value signal S151 is input to the subtracter101 while the input shape signal S161 is input to the encoder (for shapesignals) 111.

The subtracter 101 receives the input pixel value signal S151 and thereference pixel value signal S155 output from the second switchingcircuit 106, and outputs the difference pixel value signal S152 obtainedby the subtracting process to the encoder 102. The encoder 102 subjectsthe difference pixel value signal S152 to the predetermined compressioncoding process to generate the coded pixel value signal S153. The codedpixel value signal S153 becomes part of the output of the image codingapparatus according to the first embodiment while it is input to thedecoder 103. The decoder 103 subjects the received coded pixel valuesignal S153 to the decoding process, which is the converse process ofthe coding process of the encoder 102, to generate the coded and thendecoded pixel value signal S154. The coded and then decoded pixel valuesignal S154 is output to the adder 104.

The adder 104 receives the coded and then decoded pixel value signalS154 and the reference pixel value signal S155 output from the secondswitching circuit 106, and outputs the decoded pixel value signal S156obtained by the adding process to the first switching circuit 105.

The first switching circuit 105 outputs the input decoded pixel valuesignal S156 to the memory 107 or 108 according to the decoding pixelvalue changing signal S171. The memories 107 and 108 retain the inputdecoded pixel value signal S156 in flames. The first switching circuit105 abandons the decoded pixel value signal S156 not to be output to anymemories when the decoding pixel value changing signal S171 instructsthe first switching circuit 105 not to retain the decoded pixel valuesignal S156.

The prediction changing unit 110 instructs the first switching circuit105 through the decoding pixel value changing signal S171 to output thedecoded pixel value signal S156 alternately to the memory 107 and to thememory 108 in principle so that the decoded pixel value signal S156 isoutput to either memory different from the other memory into which thepixel value signal S156 is previously output. The decoding pixel valuechanging signal S171 also instructs the first switching circuit 105 toabandon the decoded pixel value signal S156 when the decoded pixel valuesignal S156 is a signal which is not to be referred in the codingprocess.

The decoded pixel value signal S156 input to the memory 107 or 108 isretained as the first decoded and then stored pixel value signal S157 orthe second decoded and then stored pixel value signal S158,respectively. The average calculator 109 receives the first decoded andthen stored pixel value signal S157 and the second decoded and thenstored pixel value signal S158 and obtains the average of them togenerate the average decoded pixel value signal S159.

The second switching circuit 106 selects either the predetermined value,the first decoded and then stored pixel value signal S157, the seconddecoded and then stored pixel value signal S158, or the average decodedpixel value signal S159, according to the reference pixel value changingsignal S172 from the prediction changing unit 110, to output theselected signal as the reference pixel value signal S155 to thesubtracter 101 and the adder 104.

The second switching circuit 106 is instructed with the use of thereference pixel value changing signal S172 from the prediction changingunit 110 in a way that follows. When a coding object to be subjected tothe coding process in the encoder 102 is an I frame, the secondswitching circuit 106 is instructed to select the predetermined valuesince the coding is carried out without the reference process. Hence thepredetermined value which is used in the intra-frame coding is output asthe reference pixel value signal S155.

When the coding object in the encoder 102 is a P flame, the predictionchanging unit 110 instructs the second switching circuit 106 to select,among the first decoded and then stored pixel value signal S157 and thesecond decoded and then stored pixel value signal S158, the one which isat a forward position on time series from the coding object.

When the coding object in the encoder 102 is a B flame, the predictionchanging unit 110 instructs the second switching circuit 106 to selecteither the first decoded and then stored pixel value signal S157, thesecond decoded and then stored pixel value signal S158, or the averagedecoded pixel value signal S159. In the first embodiment, the predictionchanging unit 110 selects the signal that has the smallest differencewhich is to be obtained in the subtracter 101 of the signals which arepossible to be selected. Hence an image which is to make the motiondetection error of the pixel value signal the smallest is selected as areference image among an image which is earlier than an image to becoded, an image which is later than an image to be coded, or an imagewhich is obtained from the average of them.

Moreover, when the coding object is a P frame or B frame, bothintra-frame coding and inter-frame coding can be carried out, and alsothe prediction changing unit 110 can instruct the second switchingcircuit 106 to select the inter-frame coding depending on conditions andto output the predetermined value.

The above-described coding process for the input pixel value signal S151is similar to that in the case where the prior art intra-frame codingand inter-frame coding are carried out. Especially in the coding processfor the B frame, the coding efficiency is improved by selecting oneamong plural reference candidates.

On the other hand, in the image coding apparatus according to the firstembodiment, the input shape signal S161, which is the input of theapparatus, is input to the encoder (for shape signals) 111 and issubjected to a compression coding in the encoder 111. This compressioncoding is carried out with reference to the decoded shape signal S166output from the fourth switching circuit 114 which will be describedhereinafter. The encoder 111 outputs the coded shape signal S162, whichis generated by the coding process, as part of the output of the imagecoding apparatus, and the coded shape signal S162 is also input to thedecoder (for shape signals) 112.

The decoder 112 subjects the input coded shape signal S162 to thedecoding process which is the converse process of the coding process inthe encoder 111 to generate the decoded shape signal S163. This decodingprocess is carried out with reference to the reference shape signal S166output from the fourth switching circuit 114.

The decoded shape signal S163 is output to the third switch circuit 113.The third switching circuit 113 outputs the input decoded shape signalS163 to the memory 115 or the memory 116 according to the instruction ofthe decoding shape changing signal S173 output from the predictionchanging unit 110. The memories 115 and 116 contain the input decodedshape signal S163 in frames. The third switching circuit 113 abandonsthe decoded shape signal S163 not to be output to any memories when thedecoding shape changing signal S173 instructs the third switchingcircuit 113 not to retain the decoded shape signal S163.

The prediction changing unit 110 instructs the third switching circuit113 through the decoding pixel value changing signal S173 to output thedecoded shape signal S163 alternately to the memory 115 and to thememory 116 in principle so that the decoded shape signal S163 is outputto either memory different from the other memory into which the decodedshape signal S163 is previously output. The decoding shape changingsignal S173 also instructs the third switching circuit 113 to abandonthe decoded shape signal S163 when the decoded shape signal S163 is asignal which is not to be referred in the coding process. The decodedshape signal S163 input to the memory 115 or 116 is retained as thefirst decoded and then stored shape signal S164 or the second decodedand then stored shape signal S165, respectively.

The fourth switching circuit 114 selects either the predetermined value,the first decoded and then stored shape signal S164, or the seconddecoded and then stored shape signal S165, according to the referenceshape changing signal S174 from the prediction changing unit 110, tooutput the selected signal as the reference shape signal S166 to theencoder 111 and the decoder 112.

FIG. 2(a) to FIG. 2(c) are diagrams for explaining the coding processwith references for shape signals. FIG. 2(a) shows a decoded shapesignal at time t0 which is obtained by coding a shape signal which ispositioned at forward time t0 on time series from a shape signal at timet1 to be coded and then decoding the same. FIG. 2(c) shows a decodedshape signal at time t2 which is obtained by coding a shape signal whichis positioned at backward time t2 on time series from a shape signal attime t1 to be coded and then decoding the same. As described above, thecoding efficiency is not necessarily improved for the shape signal whichis two-valued information, even if a difference value between shapesignals is obtained referring to temporarily adjacent information or theaverage of plural information which are located at forward and backwardposition on time series is referred. However, there are some cases wherea process based on the temporal correlation is effective.

As shown in FIG. 2, the shape signal of an image at time t1 to be codedis partly the same as the shape signal of the decoded image at forwardtime t0 shown in FIG. 2(a) and as the shape signal of the decoded imageat backward time t2 shown in FIG. 2(c). In this case, the codingefficiency can be improved by generating a predicted shape signal withthe use of the temporarily forward and backward adjacent decoded shapesignals. To improve the precision of the prediction, a decoded shapesignal used for the prediction should be selected for each appropriateunit of a shape signal to be coded. Accordingly, in the firstembodiment, a prediction method for coding the shape signal is selectedapart from the selection of the reference image for coding the pixelvalue signal, whereby the coding efficiency is improved by the processwith reference in coding the shape signal.

In the image coding apparatus according to the first embodiment, theprediction changing unit 110 instructs the fourth switching circuit 114through the reference shape changing signal S174 to select a signalwhich is to have the smallest output bit number in the encoder 111 andoutput it as the reference shape signal S166. Accordingly, the encoder111 carries out one which has the best coding efficiency of anintra-frame coding without reference but with the use of the referenceshape signal S166 which is the predetermined value, and an inter-framecoding referring to the shape signal at forward time or backward timewith the use of the reference shape signal S166 which is the first orsecond decoded and then stored shape signal S164 or S165.

When the input pixel value signal S151 and the input shape signal S161are coded as described above, the prediction changing unit 110 generatesand outputs the decoding pixel value changing signal S171, the referencepixel value changing signal S172, the decoding shape changing signalS173 and the reference shape changing signal S174 for controlling eachswitching circuit. The prediction changing unit 110 also codes eachgenerated changing signal to generate a prediction selection signalS175. The prediction selection signal S175 is the output of the imagecoding apparatus according to the first embodiment as well as the codedpixel value signal S153 and the coded shape signal S162, all of whichare used in the decoding process.

As described above, the image coding apparatus according to the firstembodiment comprises the subtracter 101, the encoder (for pixel valuesignals) 102, the decoder (for pixel value signals) 103, the adder 104,the first switching circuit 105, the second switching circuit 106, thememories 107 and 108, the average calculator 109, the predictionchanging unit 110, the encoder (for shape signals) 111, the decoder (forshape signals) 112, the third switching circuit 113, the fourthswitching 114, and the memories 115 and 116. Therefore, the predictionchanging unit 110 controls separately and independently the selection ofthe reference pixel value signal used for coding the input pixel valuesignal S151 in the second switching circuit 106 and the selection of thereference shape signal used for coding the input shape signal S161 inthe fourth switching circuit 114, whereby the coding efficiency can beimproved for both the input pixel value signal and the input shapesignal.

In the first embodiment, the prediction changing unit 110 instructs thefourth switching circuit 114 to do the selection whereby the output ofthe encoder 111 has the smallest bit number. However this is amongexamples. The control of the fourth switching circuit 114 by theprediction changing unit 110 can be determined in another way accordingto the property of the image to be coded or the performance andprocessing status of the image coding apparatus. For example, bysupervising the free amount of the send buffer when the result of thecoding in the first embodiment is sent, the intra-frame coding gets apriority to be first carried out when the free amount is large.

Embodiment 2

An image decoding method and an image decoding apparatus according to asecond embodiment of the present invention appropriately decode theresult of the image coding process according to the first embodiment.

FIG. 3 is a block diagram showing the configuration of the imagedecoding apparatus according to the second embodiment. As shown in thefigure, the image decoding apparatus according to the second embodimentcomprises a decoder (for pixel value signals) 303, an adder 304, a firstswitching circuit 305, a second switching circuit 306, a memory (a firstmemory for retaining decoded pixel value signals) 307, a memory (asecond memory for retaining decoded pixel value signals) 308, an averagecalculator 309, a decoder (for shape signals) 312, a third switchingcircuit 313, a fourth switching circuit 314, a memory (a first memoryfor retaining shape signals) 315, a memory (a second memory forretaining shape signals) 316, a memory for rearrangement (for pixelvalue signals) 361, a memory for rearrangement (for shape signals) 362,and a prediction changing unit 370.

In the figure, the decoder (for pixel value signals) 303, the adder 304,the first switching circuit 305, the second switching circuit 306, thememories 307 and 308, and the average calculator 309 are similar to 103to 109 of the first embodiment, which serve as a pixel value signaldecoding means which decodes the input coded pixel value signalreferring to the decoded reference pixel value signal. The decoder (forshape signals) 312, the third switching circuit 313, the fourthswitching circuit 314, and the memories 315 and 316 are similar to 112to 116 of the first embodiment, which serve as a shape signal decodingmeans which decodes the input coded shape signal referring to thedecoded reference shape signal.

The memories for rearrangement 361 and 362 retain the result of decodingfor rearrangement to be required. As described using FIG. 14, when thecoding process accompanying the bidirectional reference is carried out,the image which is at a backward position on time series is first coded.Accordingly, if the input result of the coding is simply decoded andoutput according to the input order, the result of the decoding does notsometimes agree with the original image. Hence the input result of thecoding is once retained in the memories for rearrangement, and then readand output according to the correct order.

The prediction changing unit 370 decodes the input prediction selectionsignal to obtain a control signal for each switching circuit output bythe prediction changing unit 110 (FIG. 1), and based on the controlsignal, output the changing signal to each switching circuit to indicatethe reference signal for the decoding of the pixel value signal and theshape signal, whereby the prediction changing unit 370 serves as adecoded reference specification signal generating means.

A description is given of the operation of the image decoding apparatusaccording to the second embodiment so constructed, as follows.

The image decoding apparatus according to the second embodiment receivesan input coded pixel value signal S353, an input coded shape signal S363and an input prediction selection signal S375. The input coded pixelvalue signal S353, the input coded shape signal S363 and the inputprediction selection signal S375 correspond to the outputs of the imagecoding apparatus according to the first embodiment, namely, the codedpixel value signal S153, the coded shape signal S162 and the predictionselection signal S175, respectively. In the image decoding apparatusaccording to the second embodiment, the input coded pixel value signalS353 is input to the decoder (for pixel value signals) 303, the inputcoded shape signal S363 is input to the decoder (for shape signals) 312,and the input prediction selection signal S375 is input to theprediction changing unit 370.

The prediction changing unit 370 subjects the input prediction selectionsignal S375 to the decoding process to obtain either a decoding pixelvalue changing signal, a reference pixel value changing signal, adecoding shape changing signal, or a reference shape changing signal.The prediction changing unit 370 outputs a decoding pixel value changingsignal S371, a reference pixel value changing signal S372, a decodingshape changing signal S373 and a reference shape changing signal S374 tothe first to fourth switching circuit, depending on the obtained signal.

The decoder (for pixel value signals) 303 subjects the coded pixel valuesignal S353 to the decoding process to generate a coded and then decodedpixel value signal S321 which is output to the adder 304. The adder 304receives a reference pixel value signal S323 and adds the coded and thendecoded pixel value signal S321 to the reference pixel value signal S323to generate a decoded pixel value signal S322. The decoded pixel valuesignal S322 is input and retained in the memory for rearrangement 361while being output to the first switching circuit 305.

The first switching circuit 305, similarly to the first embodiment,switches the output destinations of the decoded pixel value signal S322according to the decoding pixel value changing signal S371 output fromthe prediction changing unit 370. Hence the decoded pixel value signalS322 is retained either in either of the memories or in neither of thememories. The average calculator 309 receives the signals, which areretained in the memories 307 and 308, obtains the average of the signalsand then generates an average decoded pixel value signal S326.

The second switching circuit 306, similarly to the first embodiment,selects a signal used as the reference pixel value signal S323 accordingto the reference pixel value changing signal S372 output from theprediction changing unit 370. According to the selection, the secondswitching circuit 306 outputs either a predetermined value, a firstdecoded and then stored pixel value signal S324 retained in the memory307, a second decoded and then stored pixel value signal S325 retainedin the memory 308, or an average decoded pixel value signal S326, to theadder 304 as the reference pixel value signal S323.

As described above, the decoding pixel value changing signal S371 andthe reference pixel value changing signal S372 output by the predictionchanging unit 370 are the same as those output by the image codingapparatus according to the first embodiment. Similarly to the firstembodiment, the selection of the first switching circuit 305 executesthe storage of the memories 307 and 308. Similarly to the firstembodiment, the selection of the second switching circuit 306 makes thereference signal similar to that employed in the coding process employedin the decoding process.

On the other hand, the decoder (for shape signals) 312 decodes the inputcoded shape signal S363 using a reference shape signal S332 input fromthe fourth switching circuit 314 to generate a decoded shape signalS331. The decoded shape signal S331 is retained in the memory forrearrangement 362 while being output to the third switching circuit 313.The third switching circuit 313, similarly to the first embodiment,switches the output destinations of the decoded shape signal S331according to the decoding shape changing signal S373. Hence the decodedshape signal S331 is retained either in either of the memories or inneither of the memories.

The fourth switching circuit 314, similarly to the first embodiment,selects a signal used as the reference shape signal S332 according tothe reference shape changing signal S374 output from the predictionchanging unit 370. According to the selection, either a predeterminedvalue, a first decoded and then stored shape signal S333 retained in thememory 315 or a second decoded and then stored shape signal S334retained in the memory 316 is output as the reference shape signal S332from the fourth switching circuit 314 to the decoder (for shape signals)312.

As described above, the decoding shape changing signal S373 and thereference shape changing signal S374 output by the prediction changingunit 370 are the same as those output by the image coding apparatusaccording to the first embodiment. Similarly to the first embodiment,the selection of the third switching circuit 313 executes the storage ofthe memories 315 and 316. Similarly to the first embodiment, theselection of the fourth switching circuit 314 makes the reference signalsimilar to that employed in the coding process employed in the decodingprocess.

The decoded pixel value signal S322 processed using the appropriatereference pixel value signal S323, and the decoded shape signal S331processed using the appropriate shape signal S332 each are retained inthe memories for rearrangement 361 and 362, respectively. The imagedecoding apparatus outputs the result of decoding a pixel value signal,S381, and the result of decoding a shape signal, S382.

The image decoding apparatus according to the second embodimentcomprises the decoder (for pixel value signals) 303, the adder 304, thefirst switching circuit 305, the second switching circuit 306, thememories 307 and 308, the average calculator 309, the decoder (for shapesignals) 312, the third switching circuit 313, the fourth switchingcircuit 314, the memories 315 and 316, the memory for rearrangement (forpixel value signals) 361, the memory for rearrangement (for shapesignals) 362, and the prediction changing unit 370. The predictionchanging unit 370 employs the control signals based on the size obtainedby decoding the input prediction selection signal S375 to instruct eachswitching circuit for its selection. Therefore, both the input codedpixel value signal S353 and the input coded shape signal S363 which havebeen efficiently coded by the image coding apparatus according to thefirst embodiment can be appropriately decoded.

Embodiment 3

An image coding method and an image coding apparatus according to athird embodiment of the present invention control the pixel value signalprocess and the shape signal process, similarly to the image codingprocess according to the first embodiment.

The configuration of the image coding apparatus according to the thirdembodiment is similar to that in the first embodiment. Hence FIG. 1 isused for the explanation. Also, in the operation of the image codingapparatus according to the third embodiment, the coding process for thepixel value signal and the shape signal is carried out similarly to thatin the first embodiment.

The image coding apparatus according to the third embodiment differsfrom that according to the first embodiment in the method of generatingthe prediction selection signal S175 by the prediction changing unit110. FIG. 4(a) and FIG. 4(b) are diagrams for explaining the method ofcode assignment when the prediction selection S175 is generated in theimage coding apparatus according to the third embodiment. A descriptionis given of the generation method of the prediction selection signalS175 by the prediction changing unit 110 according to the thirdembodiment, referring to FIG. 4, as follows.

FIG. 4(a) shows the code assignment in the shape signal processing. InFIG. 1, the prediction changing unit 110 generates the predictionselection signal S175, which is a code ‘0’, ‘10’, or ‘11’, assignedaccording to what the prediction changing unit 110 instructs the fourthswitching circuit 114 using the reference shape changing signal S174,namely, ‘predetermined value reference’ which indicates the coding withthe predetermined value, ‘forward reference’ which indicates the codingwith reference to an image at a forward position on time series, or‘backward reference’ which indicates the coding with reference to animage at a backward position on time series.

FIG. 4(b) shows the code assignment in the pixel value signalprocessing. In FIG. 1, the prediction changing unit 110 generates theprediction selection signal S175, which is a code ‘00’, ‘01’, ‘10’, or‘11’, assigned according to what the prediction changing unit 110instructs the second switching circuit 106 using the reference pixelvalue changing signal S172, namely, ‘predetermined value reference’which indicates the coding with the predetermined value, ‘forwardreference’ which indicates the coding with reference to an image at aforward position on time series, ‘backward reference’ which indicatesthe coding with reference to an image at a backward position on timeseries, or ‘bidirectional reference’ which indicates the coding withreference to images at a forward and backward positions on time series.

In each case, when the predetermined value is a fixed value,‘predetermined value reference’ means the intra-frame coding.

As described above, the image coding apparatus according to the thirdembodiment has the configuration similar to that of the image codingapparatus according to the first embodiment, and the prediction changingunit 110 generates the prediction selection signal S175 based on thepredetermined code assignments each corresponding to the control signalused in the coding of the input pixel value signal and to the controlsignal used in the coding of the input shape signal. As a result,similarly to the first embodiment, each input signal is efficientlycoded, and the decoding process can be appropriately carried out byutilizing information on the reference process used in the decodingprocess.

Moreover, the code assignment shown in FIG. 4 is among examples, andvarious assignments can be employed. It is also possible to assign ashort code length to a case having the high frequency of occurrence andreduce the total bit number.

Embodiment 4

An image coding method and an image coding apparatus according to afourth embodiment of the present invention, similarly to the imagecoding process according to the third embodiment, concern about thegeneration method of the prediction selection signal.

The configuration of the image coding apparatus according to the fourthembodiment is similar to that in the first embodiment. Hence FIG. 1 isused for the explanation. Also, in the operation of the image codingapparatus according to the fourth embodiment, the coding process for thepixel value signal and the shape signal is carried out similarly to thatin the first embodiment.

The image coding apparatus according to the fourth embodiment differsfrom that in the first embodiment in the method of generating theprediction selection signal S175 by the prediction changing unit 110.FIG. 5(a) and FIG. 5(b) are diagrams for explaining the method of codeassignment when the prediction selection S175 is generated in the imagecoding apparatus according to the fourth embodiment. A description isgiven of the generation method of the prediction selection signal S175by the prediction changing unit 110 according to the fourth embodiment,referring to FIG. 5, as follows.

Although the signals for the information on the shape signal processingand the information on the pixel value signal are separately generatedin the third embodiment, the code assignment is defined for thecombination of both the pieces of information in the fourth embodiment.As shown in FIG. 5, in the fourth embodiment, when ‘predetermined valuereference’, ‘forward reference’ or ‘backward reference’ is selected inboth the pixel value signal processing and the shape signal processing,the prediction selection signal S175 is assigned the shortest codelength, while when ‘bidirectional reference’ is selected in the pixelvalue signal processing and ‘forward reference” or “backward reference”is selected in the shape signal processing, the prediction selectionsignal S175 is assigned the second shortest code length.

As described in the first embodiment, the coding process for the pixelvalue signal and the coding process for the shape signal are alsoseparately and independently controlled in the image coding apparatusaccording to the fourth embodiment, but there is generally somecorrelation between the selection of the reference signal for the pixelvalue signal and the selection of the reference signal for the shapesignal. This shows that when ‘forward reference” is selected since, forexample, either has a strong correlation with an image at a forwardposition on time series, the same selection is done for the other.

Accordingly, in the image coding apparatus according to the fourthembodiment, when S175 is generated, an event having the large frequencyof occurrence is assigned a short code length by the code assignmentwhere the correlation is taken into account, so that the code length ofthe prediction selection signal S175 is more reduced than in the case ofthe third embodiment and also the total coding efficiency can beimproved.

As described above, the image coding apparatus according to the fourthembodiment has the configuration similar to that of the image codingapparatus according to the first embodiment, and the prediction changingunit 110 generates the prediction selection signal S175 based on thepredetermined code assignment where the choices for both the signals arecombined, the code assignment corresponding to the control signal usedin the coding of the input pixel value signal and to the control signalused in the coding of the input shape signal. As a result, similarly tothe first embodiment, each input signal is efficiently coded, and theinformation on the reference process used in the coding process can beefficiently coded to the prediction selection signal.

Moreover, the code assignment shown in FIG. 5 is among examples, andvarious assignments can be employed. It is also possible to assign acode length corresponding to the frequency of occurrence and obtain thesimilar effects.

Embodiment 5

An image coding method and an image coding apparatus according to afifth embodiment of the present invention, similarly to the image codingprocess according to the third and fourth embodiments, concern about thegeneration method of the prediction selection signal.

The configuration of the image coding apparatus according to the fifthembodiment is similar to that in the first embodiment. Hence FIG. 1 isused for the explanation. Also, in the operation of the image codingapparatus according to the fifth embodiment, the coding process for thepixel value signal and the shape signal is carried out similarly to thatin the first embodiment.

The image coding apparatus according to the fifth embodiment differsfrom that in the first embodiment in the method of generating theprediction selection signal S175 by the prediction changing unit 110.FIG. 6(a) and FIG. 6(b) are diagrams for explaining the method of codeassignment when the prediction selection S175 is generated in the imagecoding apparatus according to the fifth embodiment. A description isgiven of the generation method of the prediction selection signal S175by the prediction changing unit 110 according to the fifth embodiment,referring to FIG. 6, as follows.

FIG. 6(a) is the code assignment for the coding process of the shapesignal, which is the same as that in the third embodiment shown in FIG.4(a). FIG. 6(b) is the code assignment for the combination of the codingprocess of the shape signal and the coding process of the pixel valuesignal.

In the fifth embodiment, to generate the prediction selection signalS175, the prediction changing unit 110 first carries out the codeassignment for the shape signal processing according to FIG. 6(a), andthen carries out the code assignment for the combination of the shapesignal processing and the pixel value signal processing according toFIG. 6(b).

For example, when ‘forward reference’ is selected for the shape signalprocessing and ‘forward reference’ is selected for the pixel valuesignal, the code ‘10’ is first assigned to the prediction selectionsignal S175 and next the code ‘0’ is assigned to the predictionselection signal S175. On the other hand, when ‘forward reference’ isselected for the shape signal processing and ‘backward reference’ isselected for the pixel value signal, the code ‘10’ is first assigned tothe prediction selection signal S175 and next the code ‘100’ is assignedto the prediction selection signal S175. FIG. 6(b), similarly to FIG. 5,assigns a short code length to a case having the high frequency ofoccurrence in view of the correlation between the shape signal and thepixel value signal.

As described above, the image coding apparatus according to the fifthembodiment has the configuration similar to that of the image codingapparatus according to the first embodiment, and the prediction changingunit 110 first carries out the code assignment according to the controlsignal used in the coding process of the input shape signal, and nextcarries out the code assignment according to the combination of thecontrol signals used in the coding process of the input shape signal andin the coding process of the input pixel value signal, and thus theprediction selection signal S175 is generated. As a result, similarly tothe first embodiment, each input signal is efficiently coded, and theinformation on the reference process used in the coding process can beefficiently coded to the prediction selection signal.

Moreover, the code assignment shown in FIG. 6 is among examplessimilarly to the third and fourth embodiments. Besides, it is alsopossible to carry out the assignment of code length according to thefrequency of occurrence and obtain the similar effects.

Embodiment 6

An image decoding method and an image decoding apparatus according to asixth embodiment of the present invention appropriately decode theresult of the coding obtained from the image coding process according tothe third embodiment.

The configuration of the image decoding apparatus according to the sixthembodiment is similar to that in the second embodiment and thereforeFIG. 3 is used for the explanation. Also, in the operation of the imagedecoding apparatus according to the sixth embodiment, the decodingprocess for the pixel value signal and the shape signal is carried outsimilarly to the second embodiment.

The image decoding apparatus according to the sixth embodiment receivesthe result of the coding output from the image coding apparatusaccording to the third embodiment. The image decoding apparatusaccording to the sixth embodiment receives a signal treated with thecode assignment shown in FIG. 4 as the input prediction selection signalS375 (FIG. 3). In the image decoding apparatus according to the sixthembodiment, the prediction changing unit 370 decodes the inputprediction selection signal S375 appropriately, whereby the result ofthe coding according to the third embodiment is appropriately decodedsimilarly to the second embodiment.

As described above, the image decoding apparatus according to the sixthembodiment has the configuration similar to the image decoding apparatusaccording to the second embodiment, and receives the result of thecoding according to the third embodiment, and the prediction changingunit 370 decodes the input prediction selection signal S375, whereby theresult of the coding according to the third embodiment can beappropriately decoded.

Embodiment 7

An image decoding method and an image decoding apparatus according to aseventh embodiment of the present invention appropriately decode theresult of the coding obtained from the image coding process according tothe fourth embodiment.

The configuration of the image decoding apparatus according to theseventh embodiment is similar to that in the second embodiment andtherefore FIG. 3 is used for the explanation. Also, in the operation orthe image decoding apparatus according to the seventh embodiment, thedecoding process for the pixel value signal and the shape signal iscarried out similarly to the second embodiment.

The image decoding apparatus according to the seventh embodimentreceives the result of the coding output from the image coding apparatusaccording to the fourth embodiment. The image decoding apparatusaccording to the seventh embodiment receives a signal treated with thecode assignment shown in FIG. 5 as the input prediction selection signalS375 (FIG. 3). In the image decoding apparatus according to the seventhembodiment, the prediction changing unit 370 decodes the inputprediction selection signal S375 appropriately, whereby the result ofthe coding according to the fourth embodiment is appropriately decodedsimilarly to the second embodiment.

As described above, the image decoding apparatus according to theseventh embodiment has the configuration similar to the image decodingapparatus according to the second embodiment, and receives the result ofthe coding according to the fourth embodiment, and the predictionchanging unit 370 decodes the input prediction selection signal S375,whereby the result of the coding according to the fourth embodiment canbe appropriately decoded.

Embodiment 8

An image decoding method and an image decoding apparatus according to aneighth embodiment of the present invention decode the result of thecoding obtained from the image coding process according to the fifthembodiment.

The configuration of the image decoding apparatus according to theeighth embodiment is similar to that in the second embodiment andtherefore FIG. 3 is used for the explanation. Also, in the operation ofthe image decoding apparatus according to the eighth embodiment, thedecoding process for the pixel value signal and the shape signal iscarried out similarly to the second embodiment.

The image decoding apparatus according to the eighth embodiment receivesthe result of the coding output from the image coding apparatusaccording to the fifth embodiment. The image decoding apparatusaccording to the eighth embodiment receives a signal treated with thecode assignment shown in FIG. 6 as the input prediction selection signalS375 (FIG. 3). In the image decoding apparatus according to the eighthembodiment, the prediction changing unit 370 decodes the inputprediction selection signal S375 appropriately, whereby the result ofthe coding according to the fifth embodiment is appropriately decodedsimilarly to the second embodiment.

As described above, the image decoding apparatus according to the eighthembodiment has the configuration similar to the image decoding apparatusaccording to the second embodiment, and receives the result of thecoding according to the fifth embodiment, and the prediction changingunit 370 decodes the input prediction selection signal S375, whereby theresult of the coding according to the fifth embodiment can beappropriately decoded.

Embodiment 9

An image coding method and an image coding apparatus according to aninth embodiment of the present invention carry out a control similar tothat in the first embodiment, but using different units of the switchingfor the pixel value signal and the shape signal.

The configuration of the image coding apparatus according to the ninthembodiment is similar to that in the first embodiment and therefore FIG.1 is used for the explanation. Also, in the operation of the imagecoding apparatus according to the ninth embodiment, the coding processfor the pixel value signal and the shape signal is carried out similarlyto the first embodiment.

In the image coding apparatus according to the ninth embodiment, theoutput of the control signal of the prediction changing unit 110 whenthe pixel value signal is controlled is different from that when theshape signal is controlled.

FIG. 7(a) and FIG. 7(b) are diagrams for explaining the units forcontrolling the switching in the ninth embodiment. FIG. 7(a) is adiagram for describing the reference process for the pixel value signal.As shown in the figure, a frame (a picture) of pixel value signalcomprises a plurality of blocks which are the units. This case has thenine blocks. The coding process is carried out for each block. In theninth embodiment, the control for the pixel value signal is changed foreach block. That is, the second switching circuit 106 in FIG. 1 carriesout the switching for each block of the input pixel value signal to becoded.

As opposed to this, for the shape signal, a frame is the unit forswitching the controls. Thus, the fourth switching circuit 114 carriesout the switching for each frame of the input shape signal to be coded.

For the pixel value signal, as shown in FIG. 7(a), when the switching ofthe reference image is carried out for each block, the predictionprecision is improved more possibly than when it is carried out for eachframe, and the coding efficiency can be generally improved. As opposedto this, as the shape signal differs from the pixel value signal in thestatistical properties, the prediction precision is not usually improvedeven if the prediction images are switched in small units. This isbecause the shape signal has different properties from the pixel valuesignal which consists of the signals which each are almost equallysignificant, that is, a signal indicating the contour of an object issignificant for the shape signal while signals indicating the outside ofthe contour and the part which is completely included inside of thecontour are not very significant.

On the other hand, for the shape signal as well as the pixel valuesignal, the more amount of code contained in the prediction selectionsignal S175 increases, the smaller units the switching is carried outin. Hence, for the shape signal, if the switching is carried out in aslarge units as that does not influence the prediction precision, theamount of code of the prediction selection signal S175 included in theoutput of the apparatus is reduced, whereby the total code efficiencycan be improved. In the particular case of an apparatus such as aportable remote terminal which sends and receives images, data and thelike by an extremely low bit rate coding, since the amount of code whichcan be assigned to the pixel value and shape signals constituting animage is small, the amount of code of the prediction selection signal isrelatively large and therefore the reduction of the amount of code haslarge effect.

As described above, the image coding apparatus according to the ninthembodiment has the configuration similar to that of the image codingapparatus according to the first embodiment, and the prediction changingunit 110 uses the blocks for the pixel value signal and the frames forthe shape signal as the units for controlling the switching, that is,the shape signal where the prediction precision is not influenced muchby the size of the unit of the switching is controlled using arelatively large unit, whereby the amount of code of the predictionselection signal S175 which is the output of the apparatus is reducedand therefore the total coding efficiency can be improved. The apparatusis suitable to the case where the process is carried out with anextremely low bit rate.

Further in the ninth embodiment, although the coding process for theshape signal always uses the frame as the unit of the switching, it ispossible to use different units of the switching such as both the frameand the block, or to select one of the different units. In this case,the prediction selection signal S175 can contain different hierarchicallevels of information such as information of a frame level andinformation of a block level. For those pieces of information, forexample, the information of the frame level specifies the referencemethod and the information of the block level specifies the intra-framecoding or the coding accompanying the reference, which can be realizedby means of the setting of the code assignment and the like.

In the ninth embodiment, for the shape signal, the controls are switchedfor each frame. This is among examples. The switching can be alsocontrolled, for example, for each macro-blocks or for each group of moreblocks. In general, if the pixel value signal and the shape signal eachuse different units for controlling the switching, and the shape signaluses the larger unit as the unit of the switching, the similar effect isobtained.

Embodiment 10

An image coding method and an image coding apparatus according to atenth embodiment of the present invention carry out a control, similarlyto that of the first embodiment, using different units of the switchingfor the pixel value signal and the shape signal.

The configuration of the image coding apparatus according to the tenthembodiment is similar to that in the first embodiment and therefore FIG.1 is used for the explanation. Also, in the operation of the imagecoding apparatus according to the tenth embodiment, the coding processfor the pixel value signal and the shape signal is carried out similarlyto the first embodiment.

In the image coding apparatus according to the tenth embodiment, theoutput of the control signal of the prediction changing unit 110 whenthe pixel value signal is controlled is different from that when theshape signal is controlled. In the tenth embodiment, for the processingof the pixel value signal the controls are switched for each blocksimilarly to the ninth embodiment. The processing of the shape signaluses a different unit for switching the controls from that in the ninthembodiment.

FIG. 8 is a diagram for explaining the unit for switching the controlsin the shape signal processing in the tenth embodiment. In the figure,frames 804 to 807 are decoded shape signals which are located closely ata forward and backward positions on time series from a shape signal tobe coded. The frames 804 to 807 are four pieces of frames positioned attime t1, time t2, time t3 and time t4. In FIG. 1, the memories 115 and116 each contain the two frames. A frame of shape signal to be codedcontains three slices in the tenth embodiment. For example, the frame807 contains the slices 8071 to 8073.

The slice 801 is contained in a shape signal frame at time te which isto be coded. The slice 801 contains a block to be coded indicated inslant lines. The control of the switching for the shape signal in thetenth embodiment uses a hierarchical unit which consisting of twolayers, i.e., a frame and a slice. In the tenth embodiment, two framesare first selected from four frames 804 to 807. A slice 802 and a slice803, which are positioned where the slice 801 is positioned, areobtained from the selected two frames, respectively. Thereafter, eitherthe slice 802, the slice 803 or a predetermined value is selected andused as the reference signal.

In FIG. 1, the prediction changing unit 110 instructs the fourthswitching circuit 114 what to select using the reference shape changingsignal S174. When the slice 802 or slice 803 in FIG. 8 is selected, thedata of the corresponding part is read from the memory 105 or the memory106 to be used as the reference shape signal S166.

As described above, the image coding apparatus according to the tenthembodiment has the configuration similar to that of the image codingapparatus according to the first embodiment. In the prediction changingunit 110, the units for controlling the switching are the block for thepixel value signal, and the frame consisting of a large number of blocksand the slice consisting of a small number of blocks for the shapesignal, whereby for the shape signal where the prediction precision isnot influenced much by the size of the unit, the controls are switchedfor each relatively large unit, and therefore, the amount of code of theprediction selection signal S175 which is the output of the apparatus isreduced and the total code efficiency can be improved. The apparatus issuitable to a case where the process is carried out with an extremelylow bit rate.

Further, in the tenth embodiment, similarly to the ninth embodiment, thecontrols using different units of the switching (hierarchical unit) arecarried out together, or one of the controls is appropriately selectedto be carried out, and the prediction selection signal can have plurallevels of information.

Still further, although the unit of the switching is described for theprocessing of the shape signal in both the ninth and tenth embodiments,the change of the unit for switching the controls is possible forprocessing of the pixel value signal. It is not always required to carryout the switching for each block.

Embodiment 11

An image decoding method and an image decoding apparatus according to aneleventh embodiment of the present invention decode the result of thecoding obtained from the image coding process according to the ninthembodiment.

The configuration of the image decoding apparatus according to theeleventh embodiment is similar to that in the second embodiment andtherefore FIG. 3 is used for the explanation. Also, in the operation ofthe image decoding apparatus according to the eleventh embodiment, thedecoding process for the pixel value signal and the shape signal iscarried out similarly to the second embodiment.

The image decoding apparatus according to the eleventh embodimentreceives the result of the coding output from the image coding apparatusaccording to the ninth embodiment. As described in the ninth embodiment,the result of the coding is obtained from the coding process where thecontrols are switched for each block of the pixel value signal and foreach frame of the shape signal. Accordingly, in the eleventh embodiment,if the prediction changing unit 370 (FIG. 3) outputs a control signalusing the appropriate unit of the switching according to the result ofthe coding, the result of the coding by the ninth embodiment can beproperly decoded.

As described above, the image decoding apparatus according to theeleventh embodiment has the configuration similar to the image decodingapparatus according to the second embodiment, and receives the result ofthe coding by the ninth embodiment, and the controls are switchedaccording to the same units as those in the ninth embodiment, wherebythe result of the coding by the ninth embodiment can be appropriatelydecoded.

Embodiment 12

An image decoding method and an image decoding apparatus according to atwelfth embodiment of the present invention decode the result of thecoding obtained from the image coding process according to the tenthembodiment.

The configuration of the image decoding apparatus according to thetwelfth embodiment is similar to that in the second embodiment andtherefore FIG. 3 is used for the explanation. Also, in the operation ofthe image decoding apparatus according to the twelfth embodiment, thedecoding process for the pixel value signal and the shape signal iscarried out similarly to the second embodiment.

The image decoding apparatus according to the twelfth embodimentreceives the result of the coding output from the image coding apparatusaccording to the tenth embodiment. As described in the tenth embodiment,the result of the coding is obtained from the coding process where thecontrols are switched for each hierarchical unit consisting of the frameand the slice for the shape signal. Accordingly, in the twelfthembodiment, if the prediction changing unit 370 (FIG. 3) outputs acontrol signal using the appropriate unit of the switching according tothe result of the coding, the result of the coding by the tenthembodiment can be properly decoded.

As described above, the image decoding apparatus according to thetwelfth embodiment has the configuration similar to the image decodingapparatus according to the second embodiment, and receives the result ofthe coding by the tenth embodiment, and the controls are switchedaccording to the same hierarchical unit as that in the tenth embodiment,whereby the result of the coding by the tenth embodiment can beappropriately decoded.

Embodiment 13

An image coding method and an image coding apparatus according to athirteenth embodiment of the present invention carry out a controlsimilar to that in the first embodiment, but the selection method of thereference signal in the coding of the shape signal is different.

The configuration of the image coding apparatus according to thethirteenth embodiment is similar to that in the first embodiment andtherefore FIG. 1 is used for the explanation. A description is given ofthe operation of the image coding apparatus according to the thirteenthembodiment.

In the image coding apparatus according to the thirteenth embodiment, amethod for controlling the selection of the reference shape signal usingthe reference shape changing signal S174 is different from that in thefirst embodiment.

In the first embodiment, when it is decided which a predetermined value,a decoded shape signal at a forward position on time series (at forwardtime), or a decoded shape signal at a backward position on time series(at backward time) is selected, the one is to be selected which has theshort bit number when it is output from the encoder (for shape signals)111. As opposed to this, in the thirteenth embodiment, when theinter-frame coding is carried out, a differences between the time of theshape signal to be coded and the forward an backward times of thedecoded signals are compared and then either which has the smallerdifference is employed.

Further, although the prediction selection signals which are to be theoutput of the apparatus are obtained from coding all the changing signaloutput from the prediction changing unit, it is also possible to useinformation on time as information indicating a reference method. In thethirteenth embodiment, the information on time is not coded, but ismade, as it is, included in the prediction selection signal.

FIG. 9(a) and FIG. 9(b) are diagrams for explaining the image codingprocess in the thirteenth embodiment. FIG. 9(a) shows the coding of thepixel value signal, which is similar to FIG. 14 used for describing theprior art. FIG. 9(b) shows the coding of the shape signal, correspondingto FIG. 9(a). In the figure, frames 910 to 916 correspond to the framesof pixel value signal 900 to 906, respectively. ‘I’, ‘P’ and ‘B’ andarrows in the figure, similarly to FIG. 14, shows the coding method andthe reference relationship. In the figure, t0 to t6 shows the time ofthe respective frames.

The pixel value signal which is input to the image coding apparatusaccording to the thirteenth embodiment is coded similarly to the priorart shown in FIG. 14. For the pixel value signal shown in FIG. 9(a), forexample, an I frame 900 at a forward position on time series and a Pframe 903 at a backward position on time series can be used as areference signal for a B frame 901. As described above, if either thedata at forward time or the data at backward time is selected, or thatboth the data are used, or that the average of both the data is obtainedand used, the prediction precision can be improved. As opposed to this,for the coding of the shape signal, even if both the forward andbackward frames are referred, the effect is not necessarily large, sothat either is referred for the coding of the shape signal in thethirteenth embodiment.

For example, for a shape signal 911 corresponding to a frame 901, adifference between time t1 and time t0 and a difference between time t1and t3 are compared and then either frame having the smaller differenceis used as a reference signal. In this case, FIG. 9(b) shows a frame 910is referred because the difference between time t0 and time t1 is thesmaller. The other reference relationships are similar.

FIG. 10 is a flowchart showing the procedure of the coding process ofthe pixel value and shape signals constituting an image signal of Bframe. The process for the frame 901 in FIG. 9(a) and the shape signal911 is described according to the flow of FIG. 10, as follows.

At step 101, time T0 of an image to be coded, a pixel value signal B0and a shape signal b0 which constitute the image to be coded, areobtained. In FIG. 9(a) and FIG. 9(b), T0 corresponds to t1, the pixelvalue signal B0 corresponds to the frame 901, and the shape signal b0corresponds to the frame 911. At step 102, times T1 and T2 of pixelvalue signals P1 and P2 which are to be used as reference signals areobtained in the coding process for the pixel value signal B0. As shownin FIG. 9(a), the frame 901 refers the frames 900 and 903, and thus T1corresponds to t0 and T2 corresponds to t3.

At step 103, differences between T0 obtained at step 101, and T1 and T2which are obtained at step 102, are calculated, and then the absolutevalues of the differences are compared. Thereafter, step 104 or step 105is executed according to the result of the comparison. In FIG. 9(a) andFIG. 9(b), the time difference between t1 and t0 is smaller than thatbetween t0 and t3, and therefore in this case, step 104 is executed.

At step 104, the shape signal to be coded b0 is coded referring to theshape signal p1 at time T1. In FIG. 9(c) and FIG. 9(b), the shape signalat time t0 which corresponds to time T1 is a shape signal 910, so thatthe shape signal 911 is coded referring to the shape signal 910.Thereafter, at step 106, a coded shape signal which is the result of thecoding is output.

At subsequent step 107, the pixel value signal B0 is coded referring tothe pixel value signals P1 and P2. As shown in FIG. 9(a), the frame 901is coded referring to the frames 900 and 903. Thereafter, at step 108, acoded pixel value signal is output as the result of the coding, and thenthe coding process for the pixel value signal of the frame is completed.

When the frame 902 of the pixel value signal and the corresponding shapesignal 912 are subjected to the coding process, step 105 is executedaccording to the judgment of step 103. In this case, the shape signal912 is coded referring to the shape signal 913.

In the procedure shown in the flow of FIG. 10, as shown in FIG. 9(b),the shape signals which are of the B frames are coded referring toeither of the signals at a forward and backward positions on time serieswhich is more temporarily close to the shape signal to be coded.

FIG. 11 is a flowchart of the coding process by the prediction changingunit 110 (FIG. 1) of the image coding apparatus according to thethirteenth embodiment. A description is given of the control in thethirteenth embodiment according to the flow of FIG. 11.

When a frame of pixel value signal and the corresponding shape signalare input, the procedure shown in FIG. 11 starts, and step 1101 judgeswhether the input image data to be coded is of the B frame or not. Ifthe input image data is not of the B frame, the flow transits to thestep 1110 where the image data is judged to be of the P frame or not.

When the image data is of the B frame, step 1102 and the following stepsare executed. First at step 1102, the prediction changing unit 110obtains time T0 of an image to be coded, and time T1 and time T2 of thepixel value signals P1 and P2 which are to be used as reference signalsin the coding process of the pixel value signal constituting the imageto be coded. Thereafter at step 1103, the absolute value of a differencebetween time T0 and time T1 and that between time T0 and time T2 arecompared, and step 1104 and 1105 are executed according to the result ofthe comparison.

In FIG. 1, a decoded shape signal based on a shape signal at time T1 isretained in either of the memories 115 and 116, while a decoded shapesignal based on a shape signal at time T2 is retained in the othermemory. When step 1104 or step 1105 is executed, a decoded shape signalat time T1 or T2 of which time is closer in time to time T0 is selectedto be used for coding a shape signal. This is carried out by theprediction changing unit 110 which outputs the reference shape changingsignal S174 to the fourth switching circuit 114 to control it in a waythat selects the first or second decoded and then stored shape signal asthe reference shape signal S166.

Thereafter, at step 1106, the prediction changing unit 110 controls theselection of a reference signal used for coding a pixel value signal. Adecoded pixel value signal at time T1 is retained in either of thememories 107 or 108 in FIG. 1, while a decoded pixel value signal attime T2 is retained in the other memory. Afterwards the averagecalculator 109 obtains an average decoded pixel value signal as anaverage of the first and second decoded and then stored pixel valuesignals. The prediction changing unit 110 outputs the reference changingsignal S172 to the second switching circuit 106 to control it in such away as that the average decoded pixel value signal S159 is output as thereference pixel value signal S155 from it.

As shown in FIG. 9, since a signal of the B frame is not referred, theprediction changing unit 110, at steps 1107 and 1108, controls not toretain a decoded signal based on the signal of the B frame in a memory.In FIG. 1, the decoded pixel value signal S156 based on the input pixelvalue signal S151 at time T0 is input to the first switching circuit105, and the decoded shape signal S163 based on the input shape signalS161 at time T0 is input to the third switching circuit 113. Theprediction changing unit 110 outputs the decoding pixel value changingsignal S171 to the first switching circuit 105, and the decoding shapechanging signal S173 to the second switching circuit 106, to control thefirst and second switching circuits in such a way as that both thedecoded pixel value signal S156 and the decoded shape signal S163 arenot input to a memory, but are abandoned.

Thereafter, step 1109 is executed. At step 1109, the prediction changingunit 110 outputs information on the reference coding for the shapesignal as the prediction selection signal S175. That is, informationindicating time P1 and time P2 are included in the prediction selectionsignal S175 without being coded, and the prediction selection S175 isoutput as the output of the image coding apparatus. When the result ofthe coding is to be decoded, the decoding is appropriately carried outusing the information indicating the time.

As step 1102 to step 1109 are executed, the image coding process for theimage data of the B frame is completed. Next, a description is given ofa case where step 1102 to step 1109 are not executed because the imagedata to be coded is judged not to be of the B frame.

Step 1110 is executed after step 1101 when it is judged whether theinput image data is of the P frame or not. When the image data is not ofthe P frame, the flow transits to step 1113 and the control of theprocessing for the I frame is executed.

When the image data is of the P frame, step 1111 to step 1112 areexecuted. At step 1111, the prediction changing unit 110 outputs thereference shape changing signal S174 to the fourth switching circuit 114to instruct it to output either of the decoded and then stored shapesignals in the memories 115 and 116 as the reference shape signal S166.The shape signal which is located at a forward position on time seriesfrom the shape signal to be coded is selected among the first and seconddecoded and then stored shape signals and output from the fourthswitching circuit 114. At step 1112, the prediction changing unit 110outputs the reference pixel value changing signal S172 to the secondswitching circuit 106 to instruct it in such a way as that either of thedecoded pixel value signals retained in the memories 107 and 108 isoutput as the reference pixel value signal S155. The pixel value signalwhich is located at a forward position on time series from the pixelvalue signal to be coded is selected among the first and second decodedand then stored pixel value signals and output from the second switchingcircuit 106.

On the other hand, when the input image data is judged not to be of theP frame at step 1110, step 1113 to step 1114 are executed. At step 1113,the prediction changing unit 110 outputs the reference changing signalS174 to the fourth switching circuit 114 to instruct it to output apredetermined value as the reference shape signal S166. Thepredetermined value which is set as the fixed value for the intra-framecoding is selected and output from the fourth switching circuit 114. Atstep 1114, the prediction changing unit 110 outputs the reference pixelvalue signal S172 to the second switching circuit 106 to instruct it tooutput a predetermined value as the reference pixel value signal S155.The predetermined value which is set as the fixed value for theintra-frame coding is selected and output from the second switchingcircuit 106.

When either the steps 1111 to 1112 or the steps 1113 to 1114 areexecuted, step 1115 to step 1116 follow step 1112 or step 1114 areexecuted, and the control of the retaining a coded signal is carriedout.

As shown in FIG. 9, as signals of the P frame and the I frame are to bereferred, the prediction changing unit 110 controls a decoded signalbased on the signals to be referred in such a way as that the decodedsignal is retained in a memory. In FIG. 1, the decoded pixel valuesignal S156 based on the input pixel value signal S151 at time T0 isinput to the first switching circuit 105, while the decoded shape signalS163 based on the input shape signal S161 at time T0 is input to thethird switching circuit 113. The prediction changing unit 110 outputsthe decoding pixel value changing signal S171 to the first switchingcircuit 105 and the decoding shape changing signal S173 to the thirdswitching circuit 113 to control them in such a way as that the decodedpixel value signal S156 and the decoded shape signal S163 are to beinput to the memory which does not receive a decoded signal previously.Each decoded signal is input and retained in the specified memory. Whenstep 1115 to step 1116 are executed, the input image signal of the Pframe or the I frame is completely processed if the decoded signal isstored in either of the memories.

As described above, the image coding apparatus according to thethirteenth embodiment has the configuration similar to that of the imagecoding apparatus according to the first embodiment, and the predictionchanging unit 110 controls the coding processes of the input pixel valuesignal and the input shape signals in such a way as that the respectiveappropriate reference methods are carried out. Therefore, each inputsignal is efficiently coded similarly to the first embodiment, and thedecoding process can be appropriately carried out using information onthe reference process in the coding.

Embodiment 14

An image coding method and an image coding apparatus according to afourteenth embodiment, similarly to the thirteenth embodiment, employ adifferent reference method for the coding of a shape signal from thatfor the coding of a pixel value signal, that is, a selection method fora reference signal is different from that in the thirteen embodiment.

The configuration of the image coding apparatus according to thefourteenth embodiment is similar to that in the first embodiment andtherefore FIG. 1 is used for the explanation. A description is given ofthe operation of the image coding apparatus according to the fourteenthembodiment.

The image coding apparatus according to the fourteenth embodiment hasthe operation almost similar to that according to the thirteenthembodiment, but a reference method when an input image signal is of theB frame is different from that in the thirteenth embodiment.

In the thirteenth embodiment, as shown in FIG. 9(b), either a shapesignal located at a forward position on time series or a shape signallocated at a backward position on time series is selected as a referencesignal for a shape signal corresponding to a pixel value signal of the Bframe. For the selection, differences in time between a shape signal tobe coded and the shape signals located at a forward and backwardpositions from the shape signal to be coded are compared, and the onewhich has the smaller difference in time is selected.

As opposed to this, in the fourteenth embodiment, the shape signallocated at a forward position on time series is always used as thereference signal. Whereas in the thirteenth embodiment shown in FIG.9(b), the shape signal 910 is referred for coding the shape signal 911,and the shape signal 913 is referred for coding the shape signal 912, inthe fourteenth embodiment, the shape signal 910 is referred both for thecoding of the shape signal 911 and for the coding of the shape signal912. Accordingly, the comparison judgment of the differences in time,which is carried out in the thirteenth embodiment, is unnecessary,whereby the control becomes simple. Especially when the time intervalsof the frames is constant, or nearly constant, the method is effective.

As described above, the image coding apparatus according to thefourteenth embodiment has the configuration similar to that of the imagecoding apparatus according to the first embodiment, and the predictionchanging unit 110 controls the coding processes of the input pixel valuesignal and the input shape signals in such a way as that the respectiveappropriate reference methods are carried out. Therefore, each inputsignal is efficiently coded similarly to the first embodiment, and thedecoding process can be appropriately carried out using information onthe reference process in the coding.

Embodiment 15

An image coding method and an image coding apparatus according to afifteenth embodiment, similarly to the thirteenth embodiment, employ adifferent reference method for the coding of a shape signal from thatfor the coding of a pixel value signal, that is, a selection method fora reference signal is different from that in the thirteen embodiment.

The configuration of the image coding apparatus according to thefifteenth embodiment is similar to that in the first embodiment andtherefore FIG. 1 is used for the explanation. A description is given ofthe operation of the image coding apparatus according to the fifteenthembodiment.

The image coding apparatus according to the fifteenth embodiment has theoperation almost similar to that according to the thirteenth embodiment,but a reference method when an input image signal is of the B frame isdifferent from those in the thirteenth and fourteenth embodiments.

In the thirteenth embodiment, as shown in FIG. 9(b), either a shapesignal at a forward position on time series or a shape signal at abackward position on time series is selected as a reference signal for ashape signal corresponding to a pixel value signal of the B frame. Forthe selection, differences in time between a shape signal to be codedand the shape signals at a forward and backward positions from the shapesignal to be coded are compared, and the one which has the smallerdifference in time is selected. Also, in the fourteenth embodiment, theshape signal at a forward position on time series is always used as areference signal.

As opposed to this, in the fifteenth embodiment, in the coding of ashape signal, the prediction changing unit 110 (FIG. 1) decides whetherthe reference method is selected by the comparison judgment such as inthe thirteenth embodiment, namely the comparison judgment selection, ora shape signal at a forward position on time series is referred, namelythe forward fixed selection. The decision by the prediction changingunit 110 can be done according to the properties of the input pixelvalue signal or the status of the coding process or the like.

As described above, the image coding apparatus according to thefifteenth embodiment has the configuration similar to that of the imagecoding apparatus according to the first embodiment, and the predictionchanging unit 110 controls the coding processes of the input pixel valuesignal and the input shape signals in such a way as that the respectiveappropriate reference methods are carried. Therefore, each input signalis, similarly to the first embodiment, efficiently coded out accordingto the properties of the input pixel value signal or the status of thecoding process or the like, and the decoding process can beappropriately carried out using information on the reference process inthe coding.

Embodiment 16

An image decoding method and an image decoding apparatus according to asixteenth embodiment appropriately decode the result of the codingobtained by the image coding process according to the thirteenthembodiment.

The image decoding apparatus according to the sixteenth embodiment hasthe configuration similar to that according to the second embodiment andtherefore FIG. 3 is used for the explanation. Also in the operation ofthe image decoding apparatus according to the sixteenth embodiment, thedecoding processes for a pixel value signal and a shape signal arealmost similar to those in the second embodiment, but a process when aninput coded signal is obtained by coding an image signal of the B frameis different from the second embodiment.

FIG. 12 is a flowchart showing the procedure of the process for a codedsignal obtained by coding an image signal of the B frame. A descriptionis given of the operation of the image decoding apparatus according tothe sixteenth embodiment referring to FIG. 3 and according to theflowchart of FIG. 12, as follows.

A coded pixel value signal, a coded shape signal and a predictionselection signal, which are output from the image coding apparatusaccording to the thirteenth embodiment, are input as a coded imagesignal including the input coded pixel value signal S353, the inputcoded shape signal S363 and the input prediction selection signal S375,and then the process starts. Initially, at step 1201, the predictionchanging unit 370 obtains the time T0 of the input coded pixel valuesignal B0 and input coded shape signal b0 to be decoded. Thereafter, atstep 1202, the prediction changing unit 370 obtains the times T1 and T2of the input decoded pixel value signals P1 and P2 which are to bereferred in a process for the input coded pixel value signal of the Bframe. At step 1203, the input coded shape signal b0 is input to thedecoder 312.

The prediction changing unit 370, at step 1204, carries out a judgmentprocess using the time obtained from step 1201 to step 1202. In thejudgment process, the absolute values of differences in time betweentime T0 and times T1 and T2 are obtained and compared, so that which isthe smaller is judged. Thereafter, when the result of the judgment showsthe absolute value of the difference in time between T0 and T1 is thesmaller, step 1205 is executed, while when the absolute value of thedifference in time between T0 and T2 is the smaller, step 1206 isexecuted.

When step 1205 is executed, the prediction changing unit 370 outputs thereference shape changing signal S374 to the fourth switching circuit 314to instruct it to output a signal p1 at time T1 among decoded shapesignals retained in the memories 315 and 316 as the reference shapesignal S332. The decoder 312 receives the reference shape signal S332from to the fourth switching circuit 314, and decodes the input codedshape signal b0 input at step 1203 referring to the reference shapesignal S332.

When step 1206 is executed, the almost similar process is carried out.The input coded shape signal b0 is decoded using a decoded shape signalp2 at time T2 as the reference shape signal S332.

When either step 1205 or step 1206 is executed, step 1207 is followedand executed, so that the decoded shape signal S331 obtained from thedecoding process is output to the memory for rearrangement 362.

Thereafter, step 1208 is executed, so that the input pixel value signalB0 is input to the decoder 303. At the next step 1209, the predictionchanging unit 370, based on information obtained from the inputprediction selection signal S375, outputs the reference pixel valuechanging signal S372 to the second switching circuit 306 to control itin such a way as that either decoded pixel value signals at times T1 andT2 retained in the memories 307 and 308, or an average decoded pixelvalue signal which is the average of both the decoded pixel valuesignals retained in the memories 307 and 308 obtained by the averagecalculator 309, is used as the reference pixel value signal S323.Thereafter, in the decoder 303, the input pixel value signal B0 input atstep 1208 is decoded referring to the reference pixel value signal S323.At step 1210, the decoded pixel value signal S322 which is generated isoutput to the memory for rearrangement 361, and then the process for theinput coded pixel value signal is completed.

As described above, the image decoding apparatus according to thesixteenth embodiment has the configuration similar to that according tothe second embodiment, and the prediction changing unit 370 controls thedecoding process for a decoded shape signal in a way that uses anappropriate reference signal, whereby the result of the coding in theimage coding apparatus according to the thirteenth embodiment can beappropriately decoded.

Embodiment 17

An image decoding method and an image decoding apparatus according to aseventeenth embodiment appropriately decode the result of the codingobtained by the image coding process according to the fourteenthembodiment.

The image decoding apparatus according to the seventeenth embodiment hasthe configuration similar to that according to the second embodiment andtherefore FIG. 3 is used for the explanation. Also in the operation ofthe image decoding apparatus according to the seventeenth embodiment,the decoding processes for a pixel value signal and a shape signal arealmost similar to those in the second embodiment, but a process when aninput coded signal is obtained by coding an image signal of the B frameis different from the second embodiment.

In this case, the image decoding apparatus according to the seventeenthembodiment operates similarly to that according to the sixteenthembodiment, but a selection method for the reference signal used fordecoding the input coded shape signal is different. In the imagedecoding apparatus according to the sixteenth embodiment, the comparisonjudgment is carried out for the selection so that a decoded shape signalhaving the smaller time interval is to be used. In the seventeenthembodiment, the comparison judgment is not carried out, but a decodedshape signal at a forward position on time series from the input codedsignal to be decoded is used as the reference shape signal. Accordingly,the result of the coding output from the image coding apparatusaccording to the fourteenth embodiment where the similar referenceprocess is carried out can be appropriately decoded.

As described above, the image decoding apparatus according to theseventeenth embodiment has the configuration similar to that accordingto the second embodiment, and the prediction changing unit 370 controlsthe decoding process for a decoded shape signal in a way that uses anappropriate reference signal, whereby the result of the coding in theimage coding apparatus according to the fourteenth embodiment can beappropriately decoded.

Embodiment 18

An image decoding method and an image decoding apparatus according to aneighteenth embodiment appropriately decode the result of the codingobtained by the image coding process according to the fifteenthembodiment.

The image decoding apparatus according to the eighteenth embodiment hasthe configuration similar to that according to the second embodiment andtherefore FIG. 3 is used for the explanation. Also in the operation ofthe image decoding apparatus according to the eighteenth embodiment, thedecoding processes for a pixel value signal and a shape signal arealmost similar to those in the second embodiment, but a process when aninput coded signal is obtained by coding an image signal of the B frameis different from the second embodiment.

In this case, the image decoding apparatus according to the eighteenthembodiment operates similarly to those according to the sixteenth andseventeenth embodiments, but a selection method for the reference signalused for decoding the input coded shape signal is different. In theimage decoding apparatus according to the sixteenth embodiment, thecomparison judgment is carried out for the selection so that a decodedshape signal having the smaller time interval is to be used. In theseventh embodiment, the comparison judgment is not carried out, but adecoded shape signal at a forward position on time series from an inputcoded signal to be decoded is used as a reference shape signal.

As opposed to this, in the eighteenth embodiment, which the comparisonjudgment or the forward fixed selection has been carried out in thecoding process is judged, and thereafter when the comparison judgmenthas been carried out, a decoding process similar to that in thesixteenth embodiment is carried out, while when the forward fixedselection has been carried out, a decoding process similar to that inthe seventeenth embodiment is carried out. Whether the comparisonjudgment has been carried out or not is judged from information includedin the prediction selection signal output by the image coding apparatusaccording to the fifteenth embodiment. Accordingly, the result of thecoding output from the image coding apparatus according to the fifteenthembodiment where the similar reference process is carried out can beappropriately decoded.

As described above, the image decoding apparatus according to theeighteenth embodiment has the configuration similar to that according tothe second embodiment, and the prediction changing unit 370 controls thedecoding process for a decoded shape signal in a way that uses anappropriate reference signal, whereby the result of the coding in theimage coding apparatus according to the fifteenth embodiment can beappropriately decoded.

Embodiment 19

An image coding program recording medium according to a nineteenthembodiment records an image coding program which carries out the imagecoding method of the present invention.

FIG. 13 shows an example of a program recording medium, i.e., a floppydisk. The image coding program recording medium according to thenineteenth embodiment is such a recording medium in which an imagecoding program carrying out any of the image coding methods shown in thefirst, third to fifth, ninth to tenth, and thirteenth to fifteenthembodiments is recorded. Accordingly, the image coding program recordingmedium according to the nineteenth embodiment can be removed, preservedand so on. The recorded image coding program can be subjected to copyingbetween the recording media and the like. The image coding apparatusshown in each embodiment can be realized by CPU and DSP and the likecarrying out the program on a computer system and the like.

As an image coding program recording medium, other than the floppy diskshown in the figure, anything which can record a program, namely anoptical disk such as a CD-ROM, a semiconductor storage such as an ICcard, and a tape medium such as a cassette tape, can be usable.

As described above, the image coding program recording medium accordingto the nineteenth embodiment can realize the image coding method andimage coding apparatus of the present invention by carrying out therecorded image coding program on a computer system and the like, andmakes the image coding method of this invention used with ease.

Embodiment 20

An image decoding program recording medium according to a twentiethembodiment records an image decoding program which carries out the imagedecoding method of the present invention.

FIG. 13 shows an example of a program recording medium, i.e., a floppydisk. The image decoding program recording medium according to thetwentieth embodiment is such a recording medium in which an imagedecoding program carrying out any of the image decoding methods shown inthe second, sixth to eighth, eleventh to twelfth, and sixteenth toeighteenth embodiments is recorded. Accordingly, the image decodingprogram recording medium according to the twentieth embodiment can bemoved, preserved and so on. The recorded image decoding program can besubjected to copying between the recording media and the like. The imagedecoding apparatus shown in each embodiment can be realized by CPU andDSP and the like carrying out the program on a computer system and thelike.

As an image decoding program recording medium, other than the floppydisk shown in the figure, anything which can record a program, namely anoptical disk such as a CD-ROM, a semiconductor storage such as an ICcard, and a tape medium such as a cassette tape, can be usable.

As described above, the image decoding program recording mediumaccording to the twentieth embodiment can realize the image decodingmethod and image decoding apparatus of the present invention by carryingout the recorded image decoding program on a computer system and thelike, and makes the image decoding method of this invention used withease.

Further, although an image signal includes a shape signal and a pixelvalue signal in the image coding processes and the image decodingprocesses described in the first to twentieth embodiments, the imagecoding methods and image coding apparatus of this invention are alsoapplicable when semitransparent objects are dealt with and a shapesignal is substituted for or combined with a transparency signal whichhas multivalued information indicating the degree of the occlusion of anobject against a background. A case where an image signal includes atransparency signal in place of a shape signal is applicable to thepresent invention if the transparency signal replaces the shape signalin the present invention. Another case where an image signal includes ashape signal, a transparency signal and a pixel value signal is alsoapplicable if the transparency signal and either the shape signal or thepixel value signal are dealt with together. By extending the applicationwhere a pixel value signal and a shape signal are separately dealt with,a transparency signal is also dealt with separately.

Still further, in the first to twentieth embodiments, assuming that airreversible coding is employed in the coding of an image signal, asignal which is decoded after being coded is used as a reference signal.However, when a reversible coding is carried out, a pixel value signaland shape signal to be coded can be used as a reference signal. In thiscase, an input signal is used as a reference signal in the codingprocess and thus an encoder is dispensable.

What is claimed is:
 1. An image coding method for coding an input imagesignal including a shape signal indicating a shape of an object and apixel value signal having information on color and brightness of saidobject, comprising: coding a pixel value signal included in an inputimage signal referring to a decoded pixel value signal obtained bydecoding a pixel value signal which has been already coded; coding ashape signal included in said input image signal referring to a decodedshape signal obtained by decoding a shape signal which has been alreadycoded; and generating a prediction selection signal having informationindicating a reference method in a coding process in said pixel valuesignal coding and said shape signal coding; wherein said shape signalcoding includes a comparison judgment wherein among a forward timedecoded shape signal which is obtained from a shape signal which islocated at a forward position on time series for a shape signal to becoded and a backward time decoded shape signal which is obtained from ashape signal which is located at a backward position on time series forsaid shape signal to be coded, said decoded shape signal closer to saidshape signal to be coded is selected, and said decoded shape signalselected in said comparison judgment is used as a reference signal. 2.The image coding method of claim 1, wherein in said shape signal coding,a forward time decoded shape signal or a backward time decoded shapesignal is used as said decoded shape reference signal, said forward timedecoded shape signal being obtained from a shape signal which is locatedat a forward position on time series for a shape signal to be coded, andsaid backward time decoded shape signal being obtained from a shapesignal which is located at a backward position on time series for saidshape signal to be coded.
 3. The image coding method of claim 2, whereinin said pixel value signal coding, at least one of a forward timedecoded pixel value signal and a backward time decoded pixel valuesignal is used as said decoded pixel value reference signal, saidforward time decoded pixel value signal being obtained from a pixelvalue signal which is located at a forward position on time series for apixel value signal to be coded, and said backward time decoded pixelvalue signal being obtained from a pixel value signal which is locatedat a backward position on time series for said pixel value signal to becoded.
 4. An image coding apparatus for coding an input image signalincluding a shape signal indicating a shape of an object and a pixelvalue signal having information on color and brightness of said object,comprising: means for coding a pixel value signal included in an inputimage signal referring to a decoded pixel value signal obtained bydecoding a pixel value signal which has been already coded; means forcoding a shape signal included in said input image signal referring to adecoded shape signal obtained by decoding a shape signal which has beenalready coded; and means for generating a prediction selection signalhaving information indicating a reference method in a coding process insaid pixel value signal coding means and said shape signal coding means;wherein said shape signal coding means includes a comparison judgmentmeans wherein among a forward time decoded shape signal which isobtained from a shape signal which is located at a forward position ontime series for a shape signal to be coded and a backward time decodedshape signal which is obtained from a shape signal which is located at abackward position on time series for said shape signal to be coded, saiddecoded shape signal closer to said shape signal to be coded isselected, and said decoded shape signal selected in said comparisonjudgment means is used as a reference signal.
 5. An image coding methodfor coding an input image signal including a shape signal indicating ashape of an object and a pixel value signal having information on colorand brightness of said object, comprising: coding a pixel value signalincluded in said input image signal referring to a decoded pixel valuesignal obtained by decoding a pixel value signal which has been alreadycoded; coding a shape signal included in said input image signalreferring to a decoded shape signal obtained by decoding a shape signalwhich has been already coded; and generating a prediction selectionsignal having information indicating a reference method in a codingprocess in said pixel value signal coding and said shape signal coding;wherein in said shape signal coding, said reference method is specifiedby referring to a forward time or a backward time decoded shape signal,per each frame; and referring to the decoded shape signal which isreferred per each frame or without referring, per each block.